How do I compare a Unicode string that has different bytes, but the same value?

by

Unicode normalization will get you there for this one:

>>> import unicodedata
>>> unicodedata.normalize("NFC", "\uf9fb") == "\u7099"
True

Use unicodedata.normalize on both of your strings before comparing them with == to check for canonical Unicode equivalence.

Character U+F9FB is a “CJK Compatibility” character. These characters decompose into one or more regular CJK characters when normalized.

More Related Contents:

  1. What is the best way to remove accents (normalize) in a Python unicode string?
  2. How to set sys.stdout encoding in Python 3?
  3. Replace non-ASCII characters with a single space
  4. Python regex matching Unicode properties
  5. Convert Unicode to ASCII without errors in Python
  6. How to correct TypeError: Unicode-objects must be encoded before hashing?
  7. Suppress the u’prefix indicating unicode’ in python strings
  8. Character reading from file in Python
  9. Python & MySql: Unicode and Encoding
  10. Facebook JSON badly encoded
  11. Python __str__ versus __unicode__
  12. Writing UTF-8 String to MySQL with Python
  13. remove unicode emoji using re in python
  14. byte string vs. unicode string. Python
  15. How to unquote a urlencoded unicode string in python?
  16. How do I specify a range of unicode characters
  17. Python ascii utf unicode
  18. matching unicode characters in python regular expressions
  19. Unicode values in strings are escaped when dumping to JSON in Python [duplicate]
  20. How to filter (or replace) unicode characters that would take more than 3 bytes in UTF-8?
  21. Displaying unicode symbols using pygame
  22. Read Unicode characters from command-line arguments in Python 2.x on Windows
  23. How to detect string byte encoding?
  24. How to encode Python 3 string using \u escape code?
  25. Python returns length of 2 for single Unicode character string
  26. Unicode encoding for filesystem in Mac OS X not correct in Python?
  27. Chinese unicode fonts in PyGame
  28. Printing Unicode in eclipse Pydev console and in Idle
  29. UnicodeDecodeError when reading CSV file in Pandas
  30. Bytes in a unicode Python string

Leave a Comment