The previously focused non-ignored token.
The (1-indexed) line containing the current token.
The character offset at which the current line begins.
The currently focused non-ignored token.
Advances the token stream to the next non-ignored token.
Looks ahead and returns the next non-ignored token, but does not change the state of Lexer.
Generated using TypeDoc
Given a Source object, creates a Lexer for that source. A Lexer is a stateful stream generator in that every time it is advanced, it returns the next token in the Source. Assuming the source lexes, the final Token emitted by the lexer will be of kind EOF, after which the lexer will repeatedly return the same EOF token whenever called.