Python the same char not equals

by

Here unicodedata.normalize might help you.

Basically if you normalize the data coming from the db, and normalize your selection to the same form, you should have a better result when using str.find, str.__contains__ (i.e. in), str.index, and friends. 

>>> u1 = chr(281)
>>> u2 = chr(101) + chr(808)
>>> print(u1, u2)
ę ę
>>> u1 == u2
False
>>> unicodedata.normalize('NFC', u2) == u1
True

NFC stands for the Normal Form Composed form. You can read up here for some description of the other possible forms.

More Related Contents:

  1. What is the best way to remove accents (normalize) in a Python unicode string?
  2. (unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape [duplicate]
  3. Unicode (UTF-8) reading and writing to files in Python
  4. Normalizing Unicode
  5. Python – ‘ascii’ codec can’t decode byte
  6. Python and BeautifulSoup encoding issues [duplicate]
  7. How to find out if Python is compiled with UCS-2 or UCS-4?
  8. NameError: global name ‘unicode’ is not defined – in Python 3
  9. UnicodeEncodeError: ‘latin-1’ codec can’t encode character
  10. MySQL “incorrect string value” error when save unicode string in Django
  11. Any gotchas using unicode_literals in Python 2.6?
  12. UnicodeEncodeError: ‘ascii’ codec can’t encode character at special name [duplicate]
  13. Python – can I detect unicode string language code?
  14. How to make unicode string with python3
  15. How do I specify a range of unicode characters
  16. Get unicode code point of a character using Python
  17. How to replace unicode characters in string with something else python?
  18. How to filter (or replace) unicode characters that would take more than 3 bytes in UTF-8?
  19. How can I check if a Python unicode string contains non-Western letters?
  20. How do convert unicode escape sequences to unicode characters in a python string
  21. Displaying unicode symbols using pygame
  22. requests.get returns 403 while the same url works in browser
  23. How do I compare a Unicode string that has different bytes, but the same value?
  24. sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings
  25. SQLite, python, unicode, and non-utf data
  26. Regex and unicode
  27. Remove non-ASCII characters from a string using python / django
  28. Unable to encode/decode pprint output
  29. How to deal with unicode string in URL in python3?
  30. Python Unicode string stored as ‘\u84b8\u6c7d\u5730’ in file, how to convert it back to Unicode?

Leave a Comment