Is this the best way to unescape unicode escape sequences in Ruby?

by

Your regex, /\u(....)/, has some problems.

First of all, \u doesn’t work the way you think it does, in 1.9 you’ll get an error and in 1.8 it will just match a single u rather than the \u pair that you’re looking for; you should use /\\u/ to find the literal \u that you want.

Secondly, your (....) group is much too permissive, that will allow any four characters through and that’s not what you want. In 1.9, you want (\h{4}) (four hexadecimal digits) but in 1.8 you’d need ([\da-fA-F]{4}) as \h is a new thing.

So if you want your regex to work in both 1.8 and 1.9, you should use /\\u([\da-fA-F]{4})/. This gives you the following in 1.8 and 1.9:

>> s="Where is \u03bc pancakes \u03BD house? And u1123!"
=> "Where is \\u03bc pancakes \\u03BD house? And u1123!"
>> s.gsub(/\\u([\da-fA-F]{4})/) {|m| [$1].pack("H*").unpack("n*").pack("U*")}
=> "Where is μ pancakes ν house? And u1123!"

Using pack and unpack to mangle the hex number into a Unicode character is probably good enough but there may be better ways.

More Related Contents:

  1. How do I remove emoji from string
  2. How to get rid of non-ascii characters in ruby
  3. Nokogiri, open-uri, and Unicode Characters
  4. How to avoid tripping over UTF-8 BOM when reading files
  5. Ruby 1.9: how can I properly upcase & downcase multibyte strings?
  6. Convert unicode codepoint to string character in Ruby
  7. How to determine if a character is a Chinese character
  8. Regex for reverse match a string
  9. Ruby, Difference between exec, system and %x() or Backticks
  10. What is the difference or value of these block coding styles in Ruby?
  11. How to understand symbols in Ruby
  12. Look up all descendants of a class in Ruby
  13. Mixing keyword with regular arguments in Ruby?
  14. Block definition – difference between braces and do-end?
  15. what’s different between each and collect method in Ruby [duplicate]
  16. Get names of all files from a folder with Ruby
  17. cannot load such file — zlib even after using rvm pkg install zlib
  18. Extract a substring from a string in Ruby using a regular expression
  19. How do you use global variables or constant values in Ruby?
  20. How to convert a string to a constant in Ruby?
  21. Super keyword in Ruby
  22. Regex to split BBCode into pieces
  23. Really Cheap Command-Line Option Parsing in Ruby
  24. Running another ruby script from a ruby script
  25. How to make an HTTP GET with modified headers?
  26. How do methods use hash arguments in Ruby?
  27. selenium 2.4.0, how to check for presence of an alert
  28. The Class/Object Paradox confusion
  29. How to turn a string into a method call? [duplicate]
  30. What is the behavior when using both positional and keyword arguments in Ruby?

Leave a Comment