Mercurial > hg > octave-nkf
diff libinterp/parse-tree/lex.h @ 16898:531473481084
rewrite string parsing to avoid unlimited lookahead
* NEWS: Mention change.
* lex.h, lex.ll (lexical_feedback::string_text,
lexical_feedback::string_line, lexical_feedback::string_column):
New data members
(lexical_feedback::lexical_feedback): Initialize them.
(lexical_feedback::reset): Initialize them.
(octave_base_lexer::begin_string): New function.
(\", "'", <COMMAND_START>[\"\']): Use begin_string to set start state
* instead of calling handle_string to parse string.
(DQ_STRING_START, SQ_STRING_START): New exclusive start states.
(<DQ_STRING_START>\"\", <DQ_STRING_START>\", <DQ_STRING_START>{NL},
<DQ_STRING_START>\\[0-7]{1,3}, <DQ_STRING_START>"\\a",
<DQ_STRING_START>"\\b", <DQ_STRING_START>"\\f",
<DQ_STRING_START>"\\n", <DQ_STRING_START>"\\r",
<DQ_STRING_START>"\\t", <DQ_STRING_START>"\\v",
<DQ_STRING_START>\\{ANY_INCLUDING_NL}, <DQ_STRING_START>[^\\\n\"]+,
<SQ_STRING_START>[^\'\n\r]*\', <SQ_STRING_START>{NL}):
New rules for parsing character strings.
(octave_base_lexer::have_continuation,
octave_base_lexer::have_ellipsis_continuation,
octave_base_lexer::handle_string): Delete.
| author | John W. Eaton <jwe@octave.org> |
|---|---|
| date | Thu, 04 Jul 2013 20:33:02 -0400 |
| parents | 11115c237231 |
| children | c702371ff6df efbe746f8fa8 |
line wrap: on
line diff
--- a/libinterp/parse-tree/lex.h +++ b/libinterp/parse-tree/lex.h @@ -280,6 +280,7 @@ looping (0), defining_func (0), looking_at_function_handle (0), block_comment_nesting_level (0), token_count (0), current_input_line (), comment_text (), help_text (), + string_text (), string_line (0), string_column (0), fcn_file_name (), fcn_file_full_name (), looking_at_object_index (), parsed_function_name (), pending_local_variables (), symtab_context (), nesting_level (), tokens () @@ -411,6 +412,13 @@ // The current help text. std::string help_text; + // The current character string text. + std::string string_text; + + // The position of the beginning of the current character string. + int string_line; + int string_column; + // Simple name of function file we are reading. std::string fcn_file_name; @@ -501,6 +509,8 @@ void prep_for_file (void); + void begin_string (int state); + virtual int fill_flex_buffer (char *buf, unsigned int max_size) = 0; bool at_end_of_buffer (void) const { return input_buf.empty (); } @@ -535,12 +545,6 @@ void finish_comment (octave_comment_elt::comment_type typ); - bool have_continuation (bool trailing_comments_ok = true); - - bool have_ellipsis_continuation (bool trailing_comments_ok = true); - - int handle_string (char delim); - int handle_close_bracket (int bracket_type); bool looks_like_command_arg (void);