Question marks in regular expressions

by

This is an excellent question, and it took me a while to see the point of the lazy ?? quantifier myself.

? – Optional (greedy) quantifier

The usefulness of ? is easy enough to understand. If you wanted to find both http and https, you could use a pattern like this:

https?

This pattern will match both inputs, because it makes the s optional.

?? – Optional (lazy) quantifier

?? is more subtle. It usually does the same thing ? does. It doesn’t change the true/false result when you ask: “Does this input satisfy this regex?” Instead, it’s relevant to the question: “Which part of this input matches this regex, and which parts belong in which groups?” If an input could satisfy the pattern in more than one way, the engine will decide how to group it based on ? vs. ?? (or * vs. *?, or + vs. +?).

Say you have a set of inputs that you want to validate and parse. Here’s an (admittedly silly) example:

Input:       
http123
https456
httpsomething

Expected result:
Pass/Fail  Group 1   Group 2
Pass       http      123
Pass       https     456
Pass       http      something

You try the first thing that comes to mind, which is this:

^(http)([a-z\d]+)$

Pass/Fail  Group 1   Group 2    Grouped correctly?
Pass       http      123        Yes
Pass       http      s456       No
Pass       http      something  Yes

They all pass, but you can’t use the second set of results because you only wanted 456 in Group 2.

Fine, let’s try again. Let’s say Group 2 can be letters or numbers, but not both:

(https?)([a-z]+|\d+)

Pass/Fail  Group 1   Group 2   Grouped correctly?
Pass       http      123       Yes
Pass       https     456       Yes
Pass       https     omething  No

Now the second input is fine, but the third one is grouped wrong because ? is greedy by default (the + is too, but the ? came first). When deciding whether the s is part of https? or [a-z]+|\d+, if the result is a pass either way, the regex engine will always pick the one on the left. So Group 2 loses s because Group 1 sucked it up.

To fix this, you make one tiny change:

(https??)([a-z]+|\d+)$

Pass/Fail  Group 1   Group 2    Grouped correctly?
Pass       http      123        Yes
Pass       https     456        Yes
Pass       http      something  Yes

Essentially, this means: “Match https if you have to, but see if this still passes when Group 1 is just http.” The engine realizes that the s could work as part of [a-z]+|\d+, so it prefers to put it into Group 2.

More Related Contents:

  1. What do ‘lazy’ and ‘greedy’ mean in the context of regular expressions?
  2. Greedy vs. Reluctant vs. Possessive Qualifiers
  3. What is the difference between .*? and .* regular expressions?
  4. Non greedy (reluctant) regex matching in sed?
  5. How can I write a regex which matches non greedy? [duplicate]
  6. Regex credit card number tests
  7. How can I make my match non greedy in vim?
  8. How to do a non-greedy match in grep?
  9. Shortest match in regex from end
  10. Learning Regular Expressions [closed]
  11. How to use Regular Expressions (Regex) in Microsoft Excel both in-cell and loops
  12. Regex to match URL
  13. Regex (grep) for multi-line search needed [duplicate]
  14. Match whitespace but not newlines
  15. Extracting a string between other two strings in R
  16. Regular expression for matching HH:MM time format
  17. How do I grep for all non-ASCII characters?
  18. How do I use regular expressions in bash scripts?
  19. Random string that matches a regexp [duplicate]
  20. Insert contents of a file after specific pattern match
  21. Regular Expression: Numeric range [duplicate]
  22. Regex: match pattern as long as it’s not in the beginning
  23. How to pattern match using regular expression in Scala?
  24. Finding the complement of a DFA?
  25. Regex: Matching by exclusion, without look-ahead – is it possible?
  26. How do I handle special characters in a Perl regex?
  27. Use gsub remove all string before first white space in R
  28. How can I find the location of a regex match in Perl?
  29. Swift 3 – How do I extract captured groups in regular expressions?
  30. Extracting text after last period in string [duplicate]

Leave a Comment