A new Gobstones lexer for a given input using Standard Gobstones.
Optional input: SourceInputwith the input to process.
PRECONDITION: there is at least one input document.
GErrors.NoInputError if there are no documents in the input.
The language options read during the source processing.
Gets the attributes read since they were last gotten, and resets the pending ones, as they are none pending.
A getter for the warnings produced during the source processing.
It returns the next proper token in the source, reading characters from it, and skipping them in the source, leaving the source ready to read the next token. If the source has no more characters, it fails.
PRECONDITION: this.hasNextGToken()
GErrors.NoMoreInputErrorIn is there is no GToken next
It returns the next token in the source, reading characters from it, and skipping them, leaving the source ready to read the next token. If the source has no more characters, it fails.
A token includes both proper tokens and filler tokens (usually whitespaces, comments, and pragmas). Filler tokens (fillers) are ignored by the parser, but this function provides all, in order to keep all characters from the input grouped into tokens.
Filler tokens that have some usefulness are properly processed (pragmas are evaluated, and their effects carried on -- whether modifying the lexer behavior or adding attributes --, comments are added as attributes).
By using this function other tools that need not to ignore filler tokens can be built. The parser should use nextGToken, that skips whitespaces and comments, and processes pragmas.
PRECONDITION: this.hasNextToken()
GErrors.NoMoreInputErrorIn is there is no Token next
Modifies the input to the given one. It resets the lexer to start the input at the beginning.
with the new input to process.
PRECONDITION: there is at least one input document.
GErrors.NoInputError if there are no documents in the input.
The subclass of BaseLexer specialized for the version of the language using the Standard Domain.