Weird backslash substitution in Ruby

by

Quick Answer

If you want to sidestep all this confusion, use the much less confusing block syntax. Here is an example that replaces each backslash with 2 backslashes:

"some\\path".gsub('\\') { '\\\\' }

Gruesome Details

The problem is that when using sub (and gsub), without a block, ruby interprets special character sequences in the replacement parameter. Unfortunately, sub uses the backslash as the escape character for these:

\& (the entire regex)
\+ (the last group)
\` (pre-match string)
\' (post-match string)
\0 (same as \&)
\1 (first captured group)
\2 (second captured group)
\\ (a backslash)

Like any escaping, this creates an obvious problem. If you want include the literal value of one of the above sequences (e.g. \1) in the output string you have to escape it. So, to get Hello \1, you need the replacement string to be Hello \\1. And to represent this as a string literal in Ruby, you have to escape those backslashes again like this: "Hello \\\\1"

So, there are two different escaping passes. The first one takes the string literal and creates the internal string value. The second takes that internal string value and replaces the sequences above with the matching data.

If a backslash is not followed by a character that matches one of the above sequences, then the backslash (and character that follows) will pass through unaltered. This is also affects a backslash at the end of the string — it will pass through unaltered. It’s easiest to see this logic in the rubinius code; just look for the to_sub_replacement method in the String class.

Here are some examples of how String#sub is parsing the replacement string:

  • 1 backslash \ (which has a string literal of "\\")

    Passes through unaltered because the backslash is at the end of the string and has no characters after it.

    Result: \

  • 2 backslashes \\ (which have a string literal of "\\\\")

    The pair of backslashes match the escaped backslash sequence (see \\ above) and gets converted into a single backslash.

    Result: \

  • 3 backslashes \\\ (which have a string literal of "\\\\\\")

    The first two backslashes match the \\ sequence and get converted to a single backslash. Then the final backslash is at the end of the string so it passes through unaltered.

    Result: \\

  • 4 backslashes \\\\ (which have a string literal of "\\\\\\\\")

    Two pairs of backslashes each match the \\ sequence and get converted to a single backslash.

    Result: \\

  • 2 backslashes with character in the middle \a\ (which have a string literal of "\\a\\")

    The \a does not match any of the escape sequences so it is allowed to pass through unaltered. The trailing backslash is also allowed through.

    Result: \a\

    Note: The same result could be obtained from: \\a\\ (with the literal string: "\\\\a\\\\")

In hindsight, this could have been less confusing if String#sub had used a different escape character. Then there wouldn’t be the need to double escape all the backslashes.

More Related Contents:

  1. Write regular expressions [closed]
  2. Get all three digit numbers from array and store in new array [closed]
  3. /^a-eg-h/.match(‘f’) gives null in Ruby
  4. Regex to match Date
  5. Difference between \A \z and ^ $ in Ruby regular expressions
  6. How to match all occurrences of a regex
  7. Ruby regular expression using variable name
  8. How do I get the match data for all occurrences of a Ruby regular expression in a string?
  9. How do I remove emoji from string
  10. Valid email address regular expression? [duplicate]
  11. Backslashes in single quoted strings vs. double quoted strings
  12. Extract a substring from a string in Ruby using a regular expression
  13. Ruby Regular expression to match a url [duplicate]
  14. How to match all occurrences of a regular expression in Ruby
  15. Regex to split BBCode into pieces
  16. What do `?i` and `?-i` in regex mean?
  17. Confusion with Atomic Grouping – how it differs from the Grouping in regular expression of Ruby?
  18. Ruby Email validation with regex
  19. What is a regex to match a string NOT at the end of a line?
  20. Match a string against multiple patterns
  21. Matching balanced parenthesis in Ruby using recursive regular expressions like perl
  22. Interpolating a string into a regex
  23. Ruby: filter array by regex?
  24. How to use double brackets in a regular expression?
  25. How to match something with regex that is not between two special characters?
  26. What is attr_accessor in Ruby?
  27. Cannot call non W3C standard command while in W3C mode (Selenium::WebDriver::Error::UnknownCommandError) with Selenium ChromeDriver in Cucumber Ruby
  28. How does “(1..4).inject(&:+)” work in Ruby
  29. Comparing ruby hashes [duplicate]
  30. What is the difference between STDIN and $stdin in Ruby?

Leave a Comment