Practical difference between parser rules and lexer rules in ANTLR?

by

… what are the practical differences between these two statements in ANTLR …

MY_RULE will be used to tokenize your input source. It represents a fundamental building block of your language.

my_rule is called from the parser, it consists of zero or more other parser rules or tokens produced by the lexer.

That’s the difference.

Do they result in different AST trees? Different performance? …

The parser builds the AST using tokens produced by the lexer, so the questions make no sense (to me). A lexer merely “feeds” the parser a 1 dimensional stream of tokens.

More Related Contents:

  1. lexers vs parsers
  2. How does the ANTLR lexer disambiguate its rules (or why does my parser produce “mismatched input” errors)?
  3. The Possibility of a “True” Quiz Generator [closed]
  4. Parse error: syntax error, unexpected T_STRING in app/design/frontend/base/default/template/catalog/product/new.phtml on line 37 [closed]
  5. Writing a parser like Flex/Bison that is usable on 8-bit embedded systems
  6. How to parse freeform street/postal address out of text, and into components
  7. What is the difference between an Abstract Syntax Tree and a Concrete Syntax Tree?
  8. Parsing command line arguments in R scripts
  9. Import CSV file with mixed data types
  10. How would you go about parsing Markdown? [closed]
  11. Using ANTLR 3.3?
  12. Handling extra operators in Shunting-yard
  13. How can error reporting in grammars be improved?
  14. How to Traverse an NLTK Tree object?
  15. Difference between constituency parser and dependency parser
  16. Looking for a clear definition of what a “tokenizer”, “parser” and “lexers” are and how they are related to each other and used?
  17. Converting a hexadecimal string to a decimal integer
  18. Reached end of file while parsing compilation error
  19. Use Scala parser combinator to parse CSV files
  20. Problem with parse a LocalDateTime using java 8
  21. How to identify whether a grammar is LL(1), LR(0) or SLR(1)?
  22. Is it a Lexer’s Job to Parse Numbers and Strings?
  23. How to manually parse a floating point number from a string
  24. Parse URL in shell script
  25. Parse clock time in java 8
  26. Python/Json:Expecting property name enclosed in double quotes
  27. Why would R use the “L” suffix to denote an integer?
  28. How can I remove extra whitespace from strings when parsing a csv file in Pandas?
  29. Simplest way to correctly load html from web page into a string in Java
  30. Parsing local XML file using Sax in Android

Leave a Comment