When is EOF needed in ANTLR 4?

by

You should include an explicit EOF at the end of your entry rule any time you are trying to parse an entire input file. If you do not include the EOF, it means you are not trying to parse the entire input, and it’s acceptable to parse only a portion of the input if it means avoiding a syntax error.

For example, consider the following rule:

file : item*;

This rule means “Parse as many item elements as possible, and then stop.” In other words, this rule will never attempt to recover from a syntax error because it will always assume that the syntax error is part of some syntactic construct that’s beyond the scope of the file rule. Syntax errors will not even be reported, because the parser will simply stop.

If instead I had the following rule:

file : item* EOF;

In means “A file consists exactly of a sequence of zero-or-more item elements.” If a syntax error is reached while parsing an item element, this rule will attempt to recover from (and report) the syntax error and continue because the EOF is required and has not yet been reached.

For rules where you are only trying to parse a portion of the input, ANTLR 4 often works, but not always. The following issue describes a technical problem where ANTLR 4 does not always make the correct decision if the EOF is omitted.

https://github.com/antlr/antlr4/issues/118

Unfortunately the performance impact of this change is substantial, so until that is resolved there will be edge cases that do not behave as you expect.

More Related Contents:

  1. When is EOF needed in ANTLR 4?
  2. What is a ‘semantic predicate’ in ANTLR?
  3. What does “fragment” mean in ANTLR?
  4. ANTLR 4.5 – Mismatched Input ‘x’ expecting ‘x’
  5. Negating inside lexer- and parser rules
  6. Visualizing an AST created with ANTLR (in a .Net environment)
  7. ANTLR What is simpliest way to realize python like indent-depending grammar?
  8. ANTLR 4 tree inject/rewrite operator
  9. ANTLR: Is there a simple example?
  10. How to create AST with ANTLR4?
  11. How to output the AST built using ANTLR?
  12. How to convert a String to its equivalent LINQ Expression Tree?
  13. If/else statements in ANTLR using listeners
  14. Practical difference between parser rules and lexer rules in ANTLR?
  15. lexers vs parsers
  16. How does the ANTLR lexer disambiguate its rules (or why does my parser produce “mismatched input” errors)?
  17. Using ANTLR 3.3?
  18. Build symbol table from grammar [closed]

Leave a Comment