Mercurial > hg > octave-nkf

annotate libinterp/parse-tree/lex.h @ 16255:12bf6a3f8c45

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
store more info in token value class * token.h, token.cc: Store token ID and trailing space info * lex.h, lex.ll (lexical_feedback::token_cache): Handle storing and retrieving extra info in the lexer.
author John W. Eaton <jwe@octave.org>
date Mon, 11 Mar 2013 14:08:50 -0400
parents a89cf57ba3a5
children b28062b977fd db7f07b22b9b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
1 /*
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
2
14138
72c96de7a403 maint: update copyright notices for 2012
John W. Eaton <jwe@octave.org>
parents: 13973
diff changeset
3 Copyright (C) 1993-2012 John W. Eaton
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
4
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
5 This file is part of Octave.
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
6
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
7 Octave is free software; you can redistribute it and/or modify it
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
8 under the terms of the GNU General Public License as published by the
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5775
diff changeset
9 Free Software Foundation; either version 3 of the License, or (at your
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5775
diff changeset
10 option) any later version.
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
11
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
12 Octave is distributed in the hope that it will be useful, but WITHOUT
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
13 ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
14 FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
15 for more details.
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
16
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
17 You should have received a copy of the GNU General Public License
7016
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5775
diff changeset
18 along with Octave; see the file COPYING. If not, see
93c65f2a5668 [project @ 2007-10-12 06:40:56 by jwe]
jwe
parents: 5775
diff changeset
19 <http://www.gnu.org/licenses/>.
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
20
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
21 */
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
22
383
0b52c68ec81f [project @ 1994-03-09 21:19:44 by jwe]
jwe
parents: 247
diff changeset
23 #if !defined (octave_lex_h)
0b52c68ec81f [project @ 1994-03-09 21:19:44 by jwe]
jwe
parents: 247
diff changeset
24 #define octave_lex_h 1
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
25
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
26 #include <deque>
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
27 #include <limits>
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
28 #include <list>
16101
8d19626b38ae provide copy contructor and operator = for lexical_feedback class.
John W. Eaton <jwe@octave.org>
parents: 16100
diff changeset
29 #include <set>
11464
21b5284fa78d avoid error when parsing nested functions
John W. Eaton <jwe@octave.org>
parents: 11367
diff changeset
30 #include <stack>
8745
6dc61981d18b better handling of object indexing in lexer
John W. Eaton <jwe@octave.org>
parents: 8701
diff changeset
31
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
32 #include "comment-list.h"
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
33 #include "input.h"
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
34 #include "token.h"
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
35
13973
2c664266e9d0 clean up parser memory on exit
John W. Eaton <jwe@octave.org>
parents: 13237
diff changeset
36 extern OCTINTERP_API void cleanup_parser (void);
2c664266e9d0 clean up parser memory on exit
John W. Eaton <jwe@octave.org>
parents: 13237
diff changeset
37
4867
c7f9ea142fda [project @ 2004-04-20 19:41:11 by jwe]
jwe
parents: 4753
diff changeset
38 // Is the given string a keyword?
c7f9ea142fda [project @ 2004-04-20 19:41:11 by jwe]
jwe
parents: 4753
diff changeset
39 extern bool is_keyword (const std::string& s);
c7f9ea142fda [project @ 2004-04-20 19:41:11 by jwe]
jwe
parents: 4753
diff changeset
40
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
41 class
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
42 stream_reader
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
43 {
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
44 public:
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
45 virtual int getc (void) = 0;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
46 virtual int ungetc (int c) = 0;
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
47
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
48 protected:
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
49 stream_reader (void) { }
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
50 ~stream_reader (void) { }
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
51
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
52 private:
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
53
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
54 // No copying!
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
55 stream_reader (const stream_reader&);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
56 stream_reader& operator = (const stream_reader&);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
57 };
16111
3ec4f6488569 move token stack to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16110
diff changeset
58
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
59 // For communication between the lexer and parser.
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
60
3585
d9803711e047 [project @ 2000-02-08 04:35:39 by jwe]
jwe
parents: 3351
diff changeset
61 class
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
62 lexical_feedback
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
63 {
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
64 public:
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
65
16106
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
66 // Did eat_whitespace or eat_continuation eat a space or tab, or a
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
67 // newline, or both?
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
68 //
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
69 // Functions that return this type will return a logical OR of the
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
70 // following values:
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
71 //
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
72 // NO_WHITESPACE no spaces to eat
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
73 // SPACE_OR_TAB space or tab in input
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
74 // NEWLINE bare new line in input
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
75
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
76 enum whitespace_type
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
77 {
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
78 NO_WHITESPACE = 1,
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
79 SPACE_OR_TAB = 2,
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
80 NEWLINE = 4
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
81 };
031117f4db7c use enum for values returned by eat_continuation and eat_whitespace
John W. Eaton <jwe@octave.org>
parents: 16105
diff changeset
82
16104
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
83 // Track nesting of square brackets, curly braces, and parentheses.
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
84
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
85 class bbp_nesting_level
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
86 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
87 private:
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
88
16107
3b791008b88e give name to anonymous enum
John W. Eaton <jwe@octave.org>
parents: 16106
diff changeset
89 enum bracket_type
16104
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
90 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
91 BRACKET = 1,
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
92 BRACE = 2,
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
93 PAREN = 3
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
94 };
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
95
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
96 public:
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
97
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
98 bbp_nesting_level (void) : context () { }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
99
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
100 bbp_nesting_level (const bbp_nesting_level& nl) : context (nl.context) { }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
101
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
102 bbp_nesting_level& operator = (const bbp_nesting_level& nl)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
103 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
104 if (&nl != this)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
105 context = nl.context;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
106
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
107 return *this;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
108 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
109
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
110 ~bbp_nesting_level (void) { }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
111
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
112 void reset (void)
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
113 {
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
114 while (! context.empty ())
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
115 context.pop ();
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
116 }
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
117
16104
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
118 void bracket (void) { context.push (BRACKET); }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
119
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
120 bool is_bracket (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
121 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
122 return ! context.empty () && context.top () == BRACKET;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
123 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
124
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
125 void brace (void) { context.push (BRACE); }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
126
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
127 bool is_brace (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
128 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
129 return ! context.empty () && context.top () == BRACE;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
130 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
131
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
132 void paren (void) { context.push (PAREN); }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
133
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
134 bool is_paren (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
135 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
136 return ! context.empty () && context.top () == PAREN;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
137 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
138
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
139 bool is_bracket_or_brace (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
140 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
141 return (! context.empty ()
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
142 && (context.top () == BRACKET || context.top () == BRACE));
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
143 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
144
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
145 bool none (void) { return context.empty (); }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
146
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
147 void remove (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
148 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
149 if (! context.empty ())
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
150 context.pop ();
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
151 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
152
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
153 void clear (void)
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
154 {
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
155 while (! context.empty ())
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
156 context.pop ();
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
157 }
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
158
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
159 private:
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
160
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
161 std::stack<int> context;
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
162 };
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
163
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
164 class token_cache
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
165 {
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
166 public:
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
167
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
168 // Store an "unlimited" number of tokens.
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
169 token_cache (size_t sz_arg = std::numeric_limits<size_t>::max ())
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
170 : buffer (), sz (sz_arg)
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
171 { }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
172
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
173 void push (token *tok)
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
174 {
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
175 if (buffer.size () == sz)
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
176 pop ();
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
177
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
178 buffer.push_front (tok);
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
179 }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
180
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
181 void pop (void)
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
182 {
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
183 if (! empty ())
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
184 {
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
185 delete buffer.back ();
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
186 buffer.pop_back ();
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
187 }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
188 }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
189
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
190 // Direct access.
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
191 token *at (size_t n)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
192 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
193 return empty () ? 0 : buffer.at (n);
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
194 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
195
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
196 const token *at (size_t n) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
197 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
198 return empty () ? 0 : buffer.at (n);
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
199 }
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
200
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
201 // Most recently pushed.
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
202 token *front (void)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
203 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
204 return empty () ? 0 : buffer.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
205 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
206
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
207 const token *front (void) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
208 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
209 return empty () ? 0 : buffer.front ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
210 }
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
211
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
212 token *back (void)
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
213 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
214 return empty () ? 0 : buffer.back ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
215 }
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
216
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
217 const token *back (void) const
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
218 {
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
219 return empty () ? 0 : buffer.back ();
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
220 }
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
221
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
222 // Number of elements currently in the buffer, max of sz.
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
223 size_t size (void) const { return buffer.size (); }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
224
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
225 bool empty (void) const { return buffer.empty (); }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
226
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
227 void clear (void)
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
228 {
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
229 while (! empty ())
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
230 pop ();
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
231 }
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
232
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
233 private:
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
234
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
235 std::deque<token *> buffer;
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
236
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
237 size_t sz;
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
238
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
239 // No copying!
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
240
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
241 token_cache (const token_cache&);
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
242
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
243 token_cache& operator = (const token_cache&);
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
244 };
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
245
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
246 lexical_feedback (void)
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
247 : end_of_input (false), convert_spaces_to_comma (true),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
248 do_comma_insert (false), at_beginning_of_statement (true),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
249 looking_at_anon_fcn_args (false), looking_at_return_list (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
250 looking_at_parameter_list (false), looking_at_decl_list (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
251 looking_at_initializer_expression (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
252 looking_at_matrix_or_assign_lhs (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
253 looking_for_object_index (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
254 looking_at_indirect_ref (false), parsing_class_method (false),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
255 maybe_classdef_get_set_method (false), parsing_classdef (false),
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
256 quote_is_transpose (false), force_script (false),
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
257 reading_fcn_file (false), reading_script_file (false),
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
258 reading_classdef_file (false),
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
259 input_line_number (1), current_input_column (1),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
260 bracketflag (0), braceflag (0),
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
261 looping (0), defining_func (0), looking_at_function_handle (0),
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
262 block_comment_nesting_level (0), token_count (0),
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
263 current_input_line (), comment_text (), help_text (),
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
264 fcn_file_name (), fcn_file_full_name (), looking_at_object_index (),
16207
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
265 parsed_function_name (), pending_local_variables (),
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
266 nesting_level (), tokens ()
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
267 {
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
268 init ();
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
269 }
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
270
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
271 ~lexical_feedback (void);
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
272
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
273 void init (void);
16194
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
274
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
275 void reset (void);
b7ca669af528 reset lexical_feedback state before starting a new parse
John W. Eaton <jwe@octave.org>
parents: 16193
diff changeset
276
16255
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
277 int previous_token_value (void) const;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
278
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
279 bool previous_token_value_is (int tok_val) const;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
280
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
281 void mark_previous_token_trailing_space (void);
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
282
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
283 bool space_follows_previous_token (void) const;
12bf6a3f8c45 store more info in token value class
John W. Eaton <jwe@octave.org>
parents: 16253
diff changeset
284
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
285 // true means that we have encountered eof on the input stream.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
286 bool end_of_input;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
287
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
288 // true means that we should convert spaces to a comma inside a
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
289 // matrix definition.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
290 bool convert_spaces_to_comma;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
291
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
292 // gag. stupid kludge so that [[1,2][3,4]] will work.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
293 bool do_comma_insert;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
294
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
295 // true means we are at the beginning of a statement, where a
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
296 // command name is possible.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
297 bool at_beginning_of_statement;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
298
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
299 // true means we are parsing an anonymous function argument list.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
300 bool looking_at_anon_fcn_args;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
301
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
302 // true means we're parsing the return list for a function.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
303 bool looking_at_return_list;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
304
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
305 // true means we're parsing the parameter list for a function.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
306 bool looking_at_parameter_list;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
307
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
308 // true means we're parsing a declaration list (global or
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
309 // persistent).
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
310 bool looking_at_decl_list;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
311
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
312 // true means we are looking at the initializer expression for a
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
313 // parameter list element.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
314 bool looking_at_initializer_expression;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
315
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
316 // true means we're parsing a matrix or the left hand side of
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
317 // multi-value assignment statement.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
318 bool looking_at_matrix_or_assign_lhs;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
319
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
320 // object index not possible until we've seen something.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
321 bool looking_for_object_index;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
322
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
323 // true means we're looking at an indirect reference to a
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
324 // structure element.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
325 bool looking_at_indirect_ref;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
326
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
327 // true means we are parsing a class method in function or classdef file.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
328 bool parsing_class_method;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
329
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
330 // true means we are parsing a class method declaration line in a
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
331 // classdef file and can accept a property get or set method name.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
332 // for example, "get.propertyname" is recognized as a function name.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
333 bool maybe_classdef_get_set_method;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
334
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
335 // true means we are parsing a classdef file
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
336 bool parsing_classdef;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
337
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
338 // return transpose or start a string?
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
339 bool quote_is_transpose;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
340
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
341 // TRUE means treat the current file as a script even if the first
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
342 // token is "function" or "classdef".
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
343 bool force_script;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
344
16199
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
345 // TRUE means we're parsing a function file.
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
346 bool reading_fcn_file;
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
347
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
348 // TRUE means we're parsing a script file.
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
349 bool reading_script_file;
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
350
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
351 // TRUE means we're parsing a classdef file.
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
352 bool reading_classdef_file;
810a71122c25 move more global variables to octave_lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16195
diff changeset
353
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
354 // the current input line number.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
355 int input_line_number;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
356
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
357 // the column of the current token.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
358 int current_input_column;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
359
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
360 // square bracket level count.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
361 int bracketflag;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
362
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
363 // curly brace level count.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
364 int braceflag;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
365
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
366 // true means we're in the middle of defining a loop.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
367 int looping;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
368
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
369 // nonzero means we're in the middle of defining a function.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
370 int defining_func;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
371
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
372 // nonzero means we are parsing a function handle.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
373 int looking_at_function_handle;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
374
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
375 // nestng level for blcok comments.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
376 int block_comment_nesting_level;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
377
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
378 // Count of tokens recognized by this lexer since initialized or
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
379 // since the last reset.
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
380 size_t token_count;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
381
16207
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
382 // The current line of input.
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
383 std::string current_input_line;
0467d68ca891 move current_input_line to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16203
diff changeset
384
16228
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
385 // The current comment text.
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
386 std::string comment_text;
e19b1632d7c1 revamp most comment handling
John W. Eaton <jwe@octave.org>
parents: 16224
diff changeset
387
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
388 // The current help text.
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
389 std::string help_text;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
390
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
391 // Simple name of function file we are reading.
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
392 std::string fcn_file_name;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
393
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
394 // Full name of file we are reading.
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
395 std::string fcn_file_full_name;
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
396
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
397 // if the front of the list is true, the closest paren, brace, or
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
398 // bracket nesting is an index for an object.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
399 std::list<bool> looking_at_object_index;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
400
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
401 // if the top of the stack is true, then we've already seen the name
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
402 // of the current function. should only matter if
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
403 // current_function_level > 0
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
404 std::stack<bool> parsed_function_name;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
405
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
406 // set of identifiers that might be local variable names.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
407 std::set<std::string> pending_local_variables;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
408
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
409 // is the closest nesting level a square bracket, squiggly brace or
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
410 // a paren?
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
411 bbp_nesting_level nesting_level;
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
412
16230
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
413 // Tokens generated by the lexer.
4bf907906134 use a queue to hold tokens in the lexer
John W. Eaton <jwe@octave.org>
parents: 16229
diff changeset
414 token_cache tokens;
16193
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
415
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
416 private:
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
417
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
418 // No copying!
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
419
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
420 lexical_feedback (const lexical_feedback&);
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
421
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
422 lexical_feedback& operator = (const lexical_feedback&);
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
423 };
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
424
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
425 // octave_lexer inherits from lexical_feedback because we will
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
426 // eventually have several different constructors and it is easier to
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
427 // intialize if everything is grouped in a parent class rather than
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
428 // listing all the members in the octave_lexer class.
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
429
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
430 class
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
431 octave_lexer : public lexical_feedback
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
432 {
d7392bf42fd1 use inheritance to simplify initialization in octave_lexer constructor
John W. Eaton <jwe@octave.org>
parents: 16185
diff changeset
433 public:
16104
c8974e28da59 move nesting_level to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16103
diff changeset
434
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
435 // Handle buffering of input for lexer.
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
436
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
437 class input_buffer
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
438 {
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
439 public:
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
440
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
441 input_buffer (void)
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
442 : buffer (), pos (0), chars_left (0), eof (false)
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
443 { }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
444
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
445 void fill (const std::string& input, bool eof_arg);
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
446
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
447 // Copy at most max_size characters to buf.
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
448 int copy_chunk (char *buf, size_t max_size);
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
449
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
450 bool empty (void) const { return chars_left == 0; }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
451
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
452 bool at_eof (void) const { return eof; }
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
453
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
454 private:
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
455
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
456 std::string buffer;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
457 const char *pos;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
458 size_t chars_left;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
459 bool eof;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
460 };
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
461
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
462 octave_lexer (void)
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
463 : lexical_feedback (), scanner (0), input_buf (), input_reader ()
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
464 {
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
465 init ();
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
466 }
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
467
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
468 octave_lexer (FILE *file)
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
469 : lexical_feedback (), scanner (0), input_buf (),
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
470 input_reader (file)
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
471 {
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
472 init ();
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
473 }
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
474
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
475 octave_lexer (const std::string& eval_string)
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
476 : lexical_feedback (), scanner (0), input_buf (),
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
477 input_reader (eval_string)
16101
8d19626b38ae provide copy contructor and operator = for lexical_feedback class.
John W. Eaton <jwe@octave.org>
parents: 16100
diff changeset
478 {
8d19626b38ae provide copy contructor and operator = for lexical_feedback class.
John W. Eaton <jwe@octave.org>
parents: 16100
diff changeset
479 init ();
8d19626b38ae provide copy contructor and operator = for lexical_feedback class.
John W. Eaton <jwe@octave.org>
parents: 16100
diff changeset
480 }
8d19626b38ae provide copy contructor and operator = for lexical_feedback class.
John W. Eaton <jwe@octave.org>
parents: 16100
diff changeset
481
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
482 ~octave_lexer (void);
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
483
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16139
diff changeset
484 void init (void);
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
485
16139
2fd39ab12209 move a function and data member from lexical_feedback to octave_parser
John W. Eaton <jwe@octave.org>
parents: 16130
diff changeset
486 void reset (void);
16122
6884401b2fbb move reset_parser lexer helper function to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16120
diff changeset
487
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
488 void prep_for_file (void);
16124
3be725cd195b move more lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16122
diff changeset
489
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
490 int read (char *buf, unsigned int max_size);
16120
4b68eb9b98b0 move octave_read lexer helper function to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16118
diff changeset
491
16164
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
492 int handle_end_of_input (void);
c5bfdc4c0963 move end_of_input flag from octve_parser class to octave_lexer class
John W. Eaton <jwe@octave.org>
parents: 16158
diff changeset
493
16128
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
494 char *flex_yytext (void);
210039e91ad6 localize use of yytext with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16126
diff changeset
495
16130
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
496 int flex_yyleng (void);
421dea028bbf localize use of yyleng with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16129
diff changeset
497
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
498 void do_comma_insert_check (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
499
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
500 int text_yyinput (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
501
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
502 void xunput (char c, char *buf);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
503
16125
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
504 void xunput (char c);
96a58f197f93 allow xunput to be called without buffer argument
John W. Eaton <jwe@octave.org>
parents: 16124
diff changeset
505
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
506 void fixup_column_count (char *s);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
507
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
508 bool inside_any_object_index (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
509
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
510 int is_keyword_token (const std::string& s);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
511
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
512 bool is_variable (const std::string& name);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
513
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
514 bool next_token_is_sep_op (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
515
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
516 bool next_token_is_postfix_unary_op (bool spc_prev);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
517
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
518 bool next_token_is_bin_op (bool spc_prev);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
519
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
520 void scan_for_comments (const char *text);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
521
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
522 int eat_whitespace (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
523
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
524 void handle_number (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
525
16253
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
526 void handle_continuation (void);
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
527
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
528 void finish_comment (octave_comment_elt::comment_type typ,
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
529 bool looking_at_continuation = false);
a89cf57ba3a5 partial cleanup of continuation handling in lexer
John W. Eaton <jwe@octave.org>
parents: 16231
diff changeset
530
16113
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
531 bool have_continuation (bool trailing_comments_ok = true);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
532
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
533 bool have_ellipsis_continuation (bool trailing_comments_ok = true);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
534
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
535 int eat_continuation (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
536
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
537 int handle_string (char delim);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
538
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
539 bool next_token_is_assign_op (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
540
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
541 bool next_token_is_index_op (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
542
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
543 int handle_close_bracket (bool spc_gobbled, int bracket_type);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
544
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
545 void maybe_unput_comma (int spc_gobbled);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
546
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
547 bool next_token_can_follow_bin_op (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
548
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
549 bool looks_like_command_arg (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
550
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
551 int handle_superclass_identifier (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
552
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
553 int handle_meta_identifier (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
554
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
555 int handle_identifier (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
556
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
557 void maybe_warn_separator_insert (char sep);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
558
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
559 void gripe_single_quote_string (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
560
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
561 void gripe_matlab_incompatible (const std::string& msg);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
562
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
563 void maybe_gripe_matlab_incompatible_comment (char c);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
564
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
565 void gripe_matlab_incompatible_continuation (void);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
566
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
567 void gripe_matlab_incompatible_operator (const std::string& op);
7c5e5e97a3bc move static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16111
diff changeset
568
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
569 void push_token (token *);
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
570
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
571 token *current_token (void);
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
572
16118
f8e463523229 move more static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16115
diff changeset
573 void display_token (int tok);
f8e463523229 move more static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16115
diff changeset
574
16129
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
575 void fatal_error (const char *msg);
053b0364b507 localize use of yy_fatal_error with lexical_feedback member function
John W. Eaton <jwe@octave.org>
parents: 16128
diff changeset
576
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
577 void lexer_debug (const char *pattern);
16118
f8e463523229 move more static lexer helper functions to lexical_feedback class
John W. Eaton <jwe@octave.org>
parents: 16115
diff changeset
578
16149
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16139
diff changeset
579 // Internal state of the flex-generated lexer.
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16139
diff changeset
580 void *scanner;
49dfba4fd3c5 use pure parser and reentrant lexer interfaces
John W. Eaton <jwe@octave.org>
parents: 16139
diff changeset
581
16183
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
582 // Object that reads and buffers input.
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
583 input_buffer input_buf;
359d56094efa handle lexer input buffering with class
John W. Eaton <jwe@octave.org>
parents: 16164
diff changeset
584
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
585 octave_input_reader input_reader;
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
586
16209
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
587 void increment_promptflag (void) { input_reader.increment_promptflag (); }
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
588
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
589 void decrement_promptflag (void) { input_reader.decrement_promptflag (); }
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
590
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
591 int promptflag (void) const { return input_reader.promptflag (); }
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
592
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
593 int promptflag (int n) { return input_reader.promptflag (n); }
e7ff32e7cf82 move global promptflag variable to octave_reader class
John W. Eaton <jwe@octave.org>
parents: 16207
diff changeset
594
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
595 std::string input_source (void) const
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
596 {
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
597 return input_reader.input_source ();
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
598 }
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
599
16203
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
600 bool input_from_terminal (void) const
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
601 {
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
602 return input_source () == "terminal";
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
603 }
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
604
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
605 bool input_from_file (void) const
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
606 {
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
607 return input_source () == "file";
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
608 }
127cccb037bf move more global parser and lexer variables to classes
John W. Eaton <jwe@octave.org>
parents: 16199
diff changeset
609
16195
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
610 bool input_from_eval_string (void) const
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
611 {
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
612 return input_source () == "eval_string";
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
613 }
b52d2f9294b6 use class for reading lexer input
John W. Eaton <jwe@octave.org>
parents: 16194
diff changeset
614
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
615 void push_start_state (int state);
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
616
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
617 void pop_start_state (void);
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
618
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
619 void clear_start_state (void);
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
620
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
621 int start_state (void) const { return start_state_stack.top (); }
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
622
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
623 void display_start_state (void) const;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
624
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
625 int handle_op (const char *pattern, int tok, bool convert = false,
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
626 bool bos = false, bool qit = false);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
627
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
628 int handle_incompatible_op (const char *pattern, int tok,
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
629 bool convert = false, bool bos = false,
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
630 bool qit = false);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
631
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
632 int handle_op_internal (const char *pattern, int tok, bool convert,
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
633 bool bos, bool qit, bool compat);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
634
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
635 int handle_token (const std::string& name, int tok);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
636
16231
2b15ae55c721 put all tokens in the token cache
John W. Eaton <jwe@octave.org>
parents: 16230
diff changeset
637 int handle_token (int tok, token *tok_val = 0);
16229
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
638
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
639 int count_token (int tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
640
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
641 int show_token (int tok);
7b7b1e4968e8 use functions instead of token return macros in lexer
John W. Eaton <jwe@octave.org>
parents: 16228
diff changeset
642
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
643 // For unwind protect.
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
644 static void cleanup (octave_lexer *lexer) { delete lexer; }
16126
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
645
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
646 private:
5c885c13bfa3 localize use of yylval.tok_val in lexical_feedback member functions
John W. Eaton <jwe@octave.org>
parents: 16125
diff changeset
647
16224
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
648 std::stack<int> start_state_stack;
4a848eb52de2 use stack for tracking lexer start states
John W. Eaton <jwe@octave.org>
parents: 16209
diff changeset
649
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16107
diff changeset
650 // No copying!
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16107
diff changeset
651
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
652 octave_lexer (const octave_lexer&);
16110
7302f8a4df83 use pointer for global lexical_feedback structure
John W. Eaton <jwe@octave.org>
parents: 16107
diff changeset
653
16158
7eb614760ddb rename lexical_feedback to octave_lexer
John W. Eaton <jwe@octave.org>
parents: 16149
diff changeset
654 octave_lexer& operator = (const octave_lexer&);
1826
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
655 };
b14829582cc4 [project @ 1996-02-02 03:05:07 by jwe]
jwe
parents: 1315
diff changeset
656
1
78fd87e624cb [project @ 1993-08-08 01:13:40 by jwe]
jwe
parents:
diff changeset
657 #endif