Lexer
class Lexer (View source)
Single state lexer class.
Lexemes can be defined on the fly. If the particular lexer instance is meant to be used with Parle\Parser, the token IDs need to be taken from there. Otherwise, arbitrary token IDs can be supplied. This lexer can give a certain performance advantage over Parle\RLexer, if no multiple states are required. Note, that Parle\RParser is not compatible with this lexer.
Constants
| ICASE |
|
| DOT_NOT_LF |
|
| DOT_NOT_CRLF |
|
| SKIP_WS |
|
| MATCH_ZERO_LEN |
|
Properties
| bool | $bol | ||
| int | $flags | ||
| int | $state | ||
| int | $marker | ||
| int | $cursor |
Methods
Processes the next rule and prepares the resulting token data.
Finalize the lexer rule set
Define token callback
Consume the data for lexing.
Dump the current state machine to stdout.
Insert a regex macro, that can be later used as a shortcut and included in other regular expressions.
Push a pattern for lexeme recognition.
Reset lexing optionally supplying the desired offset.
Details
void
advance()
Processes the next rule and prepares the resulting token data.
void
build()
Finalize the lexer rule set
Rules, previously added with Parle\Lexer::push() are finalized. This method call has to be done after all the necessary rules was pushed. The rule set becomes read only. The lexing can begin.
void
callout(int $id, callable $callback)
Define token callback
Define a callback to be invoked once lexer encounters a particular token.
void
consume(string $data)
Consume the data for lexing.
void
dump()
Dump the current state machine to stdout.
Token
getToken()
Retrieve the current token.
void
insertMacro(string $name, string $regex)
Insert a regex macro, that can be later used as a shortcut and included in other regular expressions.
void
push(string $regex, int $id)
Push a pattern for lexeme recognition.
void
reset(int $pos)
Reset lexing optionally supplying the desired offset.