Mercurial > hg > octave-nkf

annotate scripts/strings/strtok.m @ 15063:36cbcc37fdb8

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Refactor configure.ac to make it more understandable. Use common syntax for messages in config.h Correct typos, refer to libraries in all caps, use two spaces after period. Follow Autoconf guidelines and place general tests before specific tests. * configure.ac, m4/acinclude.m4: Use common syntax for messages in config.h Correct typos, refer to libraries in all caps, use two spaces after period. Follow Autoconf guidelines and place general tests before specific tests.
author Rik <rik@octave.org>
date Tue, 31 Jul 2012 10:28:51 -0700
parents 5d3a684236b0
children 333243133364
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
14138
72c96de7a403 maint: update copyright notices for 2012
John W. Eaton <jwe@octave.org>
parents: 13790
diff changeset
1 ## Copyright (C) 2000-2012 Paul Kienzle
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
2 ##
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
3 ## This file is part of Octave.
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
4 ##
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
5 ## Octave is free software; you can redistribute it and/or modify it
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
6 ## under the terms of the GNU General Public License as published by
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5827
diff changeset
7 ## the Free Software Foundation; either version 3 of the License, or (at
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5827
diff changeset
8 ## your option) any later version.
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
9 ##
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
10 ## Octave is distributed in the hope that it will be useful, but
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
11 ## WITHOUT ANY WARRANTY; without even the implied warranty of
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
12 ## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
13 ## General Public License for more details.
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
14 ##
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
15 ## You should have received a copy of the GNU General Public License
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5827
diff changeset
16 ## along with Octave; see the file COPYING. If not, see
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5827
diff changeset
17 ## <http://www.gnu.org/licenses/>.
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
18
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
19 ## -*- texinfo -*-
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
20 ## @deftypefn {Function File} {[@var{tok}, @var{rem}] =} strtok (@var{str})
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
21 ## @deftypefnx {Function File} {[@var{tok}, @var{rem}] =} strtok (@var{str}, @var{delim})
11587
c792872f8942 all script files: untabify and strip trailing whitespace
John W. Eaton <jwe@octave.org>
parents: 11523
diff changeset
22 ##
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
23 ## Find all characters in the string @var{str} up to, but not including, the
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
24 ## first character which is in the string @var{delim}. If @var{rem} is
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
25 ## requested, it contains the remainder of the string, starting at the first
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
26 ## delimiter. Leading delimiters are ignored. If @var{delim} is not
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
27 ## specified, whitespace is assumed. @var{str} may also be a cell array of
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
28 ## strings in which case the function executes on every individual string
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
29 ## and returns a cell array of tokens and remainders.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
30 ##
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
31 ## Examples:
8442
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
32 ##
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
33 ## @example
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
34 ## @group
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
35 ## strtok ("this is the life")
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
36 ## @result{} "this"
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
37 ##
8442
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
38 ## [tok, rem] = strtok ("14*27+31", "+-*/")
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
39 ## @result{}
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
40 ## tok = 14
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
41 ## rem = *27+31
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
42 ## @end group
502e58a0d44f Fix docstrings, add examples, references and tests to string functions
Thorsten Meyer <thorsten.meyier@gmx.de>
parents: 8202
diff changeset
43 ## @end example
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
44 ## @seealso{index, strsplit, strchr, isspace}
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
45 ## @end deftypefn
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
46
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
47 function [tok, rem] = strtok (str, delim)
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
48
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
49 if (nargin < 1 || nargin > 2)
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
50 print_usage ();
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
51 elseif (! (ischar (str) || iscellstr (str)))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
52 error ("strtok: STR must be a string or cell array of strings.");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
53 elseif (ischar (str) && ! isvector (str) &&! isempty (str))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
54 error ("strtok: STR cannot be a 2-D character array.");
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
55 endif
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
56
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
57 if (nargin < 2 || isempty (delim))
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
58 ws_delim = true;
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
59 else
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
60 ws_delim = false;
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
61 endif
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
62
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
63 if (isempty (str))
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
64 tok = rem = "";
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
65 elseif (ischar (str))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
66 if (ws_delim)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
67 idx = isspace (str);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
68 elseif (length (delim) <= 7)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
69 ## Build index of delimiters incrementally for low N.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
70 idx = str == delim(1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
71 for i = 2:length (delim)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
72 idx |= str == delim(i);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
73 endfor
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
74 else
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
75 ## Index the str into a mask of valid values. Faster for large N.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
76 f = false (256, 1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
77 ## This is slower than it could be because of the +1 issue.
14868
5d3a684236b0 maint: Use Octave coding conventions for cuddling parentheses in scripts directory
Rik <octave@nomad.inbox5.com>
parents: 14363
diff changeset
78 f(uint8 (delim)+1) = true;
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
79 ## Default goes via double -- unnecessarily long.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
80 si = uint32 (str);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
81 ## in-place is faster than str+1
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
82 ++si;
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
83 idx = f(si);
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
84 endif
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
85
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
86 idx_dlim = find (idx, 1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
87 idx_nodlim = find (! idx, 1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
88 if (isempty (idx_dlim))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
89 ## No delimiter. Return whole string.
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
90 tok = str;
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
91 rem = "";
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
92 elseif (idx_dlim > idx_nodlim)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
93 ## Normal case. No leading delimiters and at least 1 delimiter in STR.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
94 tok = str(1:idx_dlim-1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
95 rem = str(idx_dlim:end);
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
96 else
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
97 ## Leading delimiter found.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
98 idx_dlim = find (idx(idx_nodlim+1:end), 1);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
99 if (isempty (idx_dlim))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
100 ## No further delimiters. Return STR stripped of delimiter prefix.
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
101 tok = str(idx_nodlim:end);
10549
95c3e38098bf Untabify .m scripts
Rik <code@nomad.inbox5.com>
parents: 9036
diff changeset
102 rem = "";
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
103 else
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
104 ## Strip delimiter prefix. Return STR up to 1st delimiter
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
105 tok = str(idx_nodlim:(idx_dlim + idx_nodlim -1));
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
106 rem = str((idx_dlim + idx_nodlim):end);
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
107 endif
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
108 endif
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
109 else # Cell array of strings
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
110 if (ws_delim)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
111 delim = '\s';
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
112 endif
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
113 ptn = [ '^[' delim ']*','([^' delim ']+)','([' delim '].*)$' ];
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
114 matches = regexp (str, ptn, "tokens");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
115 eidx = cellfun ("isempty", matches);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
116 midx = ! eidx;
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
117 tok = cell (size (str));
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
118 tok(eidx) = regexprep (str(eidx), [ '^[' delim ']+' ], '');
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
119 ## Unwrap doubly nested cell array from regexp
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
120 tmp = [matches{midx}];
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
121 if (! isempty (tmp))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
122 tmp = [tmp{:}];
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
123 endif
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
124 tok(midx) = tmp(1:2:end);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
125 if (isargout (2))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
126 rem = cell (size (str));
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
127 rem(eidx) = {""};
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
128 rem(midx) = tmp(2:2:end);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
129 endif
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
130 endif
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
131
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
132 endfunction
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
133
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
134
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
135 %!demo
14237
11949c9795a0 Revamp %!demos in m-files to use Octave coding conventions on spacing, etc.
Rik <octave@nomad.inbox5.com>
parents: 14138
diff changeset
136 %! strtok ("this is the life")
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
137 %! % split at the first space, returning "this"
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
138
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
139 %!demo
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
140 %! s = "14*27+31"
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
141 %! while (1)
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
142 %! [t, s] = strtok (s, "+-*/");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
143 %! printf ("<%s>", t);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
144 %! if (isempty (s))
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
145 %! break;
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
146 %! endif
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
147 %! printf ("<%s>", s(1));
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
148 %! endwhile
14868
5d3a684236b0 maint: Use Octave coding conventions for cuddling parentheses in scripts directory
Rik <octave@nomad.inbox5.com>
parents: 14363
diff changeset
149 %! printf ("\n");
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
150 %! % ----------------------------------------------------
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
151 %! % Demonstrates processing of an entire string split on
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
152 %! % a variety of delimiters. Tokens and delimiters are
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
153 %! % printed one after another in angle brackets.
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
154
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
155 %% Test the tokens for all cases
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
156 %!assert (strtok (""), ""); # no string
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
157 %!assert (strtok ("this"), "this"); # no delimiter in string
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
158 %!assert (strtok ("this "), "this"); # delimiter at end
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
159 %!assert (strtok ("this is"), "this"); # delimiter in middle
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
160 %!assert (strtok (" this"), "this"); # delimiter at start
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
161 %!assert (strtok (" this "), "this"); # delimiter at start and end
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
162 %!assert (strtok (" "), ""(1:0)); # delimiter only
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
163
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
164 %% Test the remainder for all cases
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
165 %!test [t,r] = strtok (""); assert (r, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
166 %!test [t,r] = strtok ("this"); assert (r, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
167 %!test [t,r] = strtok ("this "); assert (r, " ");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
168 %!test [t,r] = strtok ("this is"); assert (r, " is");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
169 %!test [t,r] = strtok (" this"); assert (r, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
170 %!test [t,r] = strtok (" this "); assert (r, " ");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
171 %!test [t,r] = strtok (" "); assert (r, "");
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
172
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
173 %% Test all tokens and remainders with cell array input
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
174 %!test
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
175 %! str = {"", "this", "this ", "this is", " this", " this ", " "};
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
176 %! [t, r] = strtok (str);
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
177 %! assert (t{1}, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
178 %! assert (r{1}, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
179 %! assert (t{2}, "this");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
180 %! assert (r{2}, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
181 %! assert (t{3}, "this");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
182 %! assert (r{3}, " ");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
183 %! assert (t{4}, "this");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
184 %! assert (r{4}, " is");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
185 %! assert (t{5}, "this");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
186 %! assert (r{5}, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
187 %! assert (t{6}, "this");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
188 %! assert (r{6}, " ");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
189 %! assert (t{7}, "");
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
190 %! assert (r{7}, "");
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
191
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
192 %% Simple check for 2, 3, and 4 delimeters
14363
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
193 %!assert (strtok ("this is", "i "), "th")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
194 %!assert (strtok ("this is", "ij "), "th")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
195 %!assert (strtok ("this is", "ijk "), "th")
5827
1fe78adb91bc [project @ 2006-05-22 06:25:14 by jwe]
jwe
parents:
diff changeset
196
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
197 %% Test all cases for 8 delimiters since a different
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
198 %!# algorithm is used when more than 7 delimiters
14363
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
199 %!assert (strtok ("","jklmnop "), "")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
200 %!assert (strtok ("this","jklmnop "), "this")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
201 %!assert (strtok ("this ","jklmnop "), "this")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
202 %!assert (strtok ("this is","jklmnop "), "this")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
203 %!assert (strtok (" this","jklmnop "), "this")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
204 %!assert (strtok (" this ","jklmnop "), "this")
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
205 %!assert (strtok (" ","jklmnop "), ""(1:0))
8002
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
206
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
207 %% Test 'bad' string orientations
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
208 %!assert (strtok (" this ".'), "this".'); # delimiter at start and end
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
209 %!assert (strtok (" this ".',"jkl "), "this".');
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
210
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
211 %% Test with TAB, LF, VT, FF, and CR
8002
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
212 %!test
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
213 %! for ch = "\t\n\v\f\r"
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
214 %! [t, r] = strtok (cstrcat ("beg", ch, "end"));
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
215 %! assert (t, "beg");
14363
f3d52523cde1 Use Octave coding conventions in all m-file %!test blocks
Rik <octave@nomad.inbox5.com>
parents: 14237
diff changeset
216 %! assert (r, cstrcat (ch, "end"));
8002
30f560a5fbc3 strtok.m: include TAB, LF, VT, FF, and CR in default set of delim characters
John W. Eaton <jwe@octave.org>
parents: 7017
diff changeset
217 %! endfor
13790
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
218
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
219 %% Test input validation
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
220 %!error strtok ()
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
221 %!error strtok ("a", "b", "c")
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
222 %!error <STR must be a string> strtok (1, "b")
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
223 %!error <STR cannot be a 2-D> strtok (char ("hello", "world"), "l")
fa94d6a93d45 strtok.m: Revamp code for performance. Add cellstr input functionality.
Rik <octave@nomad.inbox5.com>
parents: 11587
diff changeset
224