When parsing Javascript, what determines the meaning of a slash?

by

It’s actually fairly easy, but it requires making your lexer a little smarter than usual.

The division operator must follow an expression, and a regular expression literal can’t follow an expression, so in all other cases you can safely assume you’re looking at a regular expression literal.

You already have to identify Punctuators as multiple-character strings, if you’re doing it right. So look at the previous token, and see if it’s any of these:

. ( , { } [ ; , < > <= >= == != === !== + - * % ++ --
<< >> >>> & | ^ ! ~ && || ? : = += -= *= %= <<= >>= >>>=
&= |= ^= / /=

For most of these, you now know you’re in a context where you can find a regular expression literal. Now, in the case of ++ --, you’ll need to do some extra work. If the ++ or -- is a pre-increment/decrement, then the / following it starts a regular expression literal; if it is a post-increment/decrement, then the / following it starts a DivPunctuator.

Fortunately, you can determine whether it is a “pre-” operator by checking its previous token. First, post-increment/decrement is a restricted production, so if ++ or -- is preceded by a linebreak, then you know it is “pre-“. Otherwise, if the previous token is any of the things that can precede a regular expression literal (yay recursion!), then you know it is “pre-“. In all other cases, it is “post-“.

Of course, the ) punctuator doesn’t always indicate the end of an expression – for example if (something) /regex/.exec(x). This is tricky because it does require some semantic understanding to disentangle.

Sadly, that’s not quite all. There are some operators that are not Punctuators, and other notable keywords to boot. Regular expression literals can also follow these. They are:

new delete void typeof instanceof in do return case throw else

If the IdentifierName you just consumed is one of these, then you’re looking at a regular expression literal; otherwise, it’s a DivPunctuator.

The above is based on the ECMAScript 5.1 specification (as found here) and does not include any browser-specific extensions to the language. But if you need to support those, then this should provide easy guidelines for determining which sort of context you’re in.

Of course, most of the above represent very silly cases for including a regular expression literal. For example, you can’t actually pre-increment a regular expression, even though it is syntactically allowed. So most tools can get away with simplifying the regular expression context checking for real-world applications. JSLint’s method of checking the preceding character for (,=:[!&|?{}; is probably sufficient. But if you take such a shortcut when developing what’s supposed to be a tool for lexing JS, then you should make sure to note that.

More Related Contents:

  1. Lexer written in Javascript? [closed]
  2. How does this code call id with html?
  3. What’s the difference between event.stopPropagation and event.preventDefault?
  4. Is it possible to send a variable number of arguments to a JavaScript function?
  5. What’s the best way to convert a number to a string in JavaScript?
  6. Cross browser JavaScript (not jQuery…) scroll to top animation
  7. How to create a dialog with “Ok” and “Cancel” options
  8. HTML5 Canvas to PNG File
  9. “var” or no “var” in JavaScript’s “for-in” loop?
  10. How to show progress bar while loading, using ajax
  11. Do objects pushed into an array in javascript deep or shallow copy?
  12. ES6 `export * from import`?
  13. Creating a select box with a search option
  14. Why is 9999999999999999 converted to 10000000000000000 in JavaScript?
  15. Copy selected text to the clipboard WITHOUT using flash – must be cross-browser
  16. How to remove global “use strict” added by babel
  17. Prevent browser from loading a drag-and-dropped file
  18. Re-associating an object with its class after deserialization in Node.js
  19. Cannot read property ‘onClicked’ of undefined when using chrome.action or chrome.browserAction
  20. Testing for multiple screens with javascript
  21. How can I duplicate a div onclick event?
  22. How to create a custom input type?
  23. Why I am not getting warnings about StrictNullChecks in typescript
  24. How can I dynamically highlight strings on a web page?
  25. Remove Datepicker Function dynamically
  26. RegEx Ignore Case
  27. for … in loop with string array outputs indices
  28. How to use pagination on HTML tables?
  29. How lazy loading images using JavaScript works?
  30. How do I add an integer value with javascript (jquery) to a value that’s returning a string?

Leave a Comment