How to read Unicode input and compare Unicode strings in Python?

by

raw_input() returns strings as encoded by the OS or UI facilities. The difficulty is knowing which is that decoding. You might attempt the following:

import sys, locale
text= raw_input().decode(sys.stdin.encoding or locale.getpreferredencoding(True))

which should work correctly in most of the cases.

We need more data about not working Unicode comparisons in order to help you. However, it might be a matter of normalization. Consider the following:

>>> a1= u'\xeatre'
>>> a2= u'e\u0302tre'

a1 and a2 are equivalent but not equal:

>>> print a1, a2
être être
>>> print a1 == a2
False

So you might want to use the unicodedata.normalize() method:

>>> import unicodedata as ud
>>> ud.normalize('NFC', a1)
u'\xeatre'
>>> ud.normalize('NFC', a2)
u'\xeatre'
>>> ud.normalize('NFC', a1) == ud.normalize('NFC', a2)
True

If you give us more information, we might be able to help you more, though.

More Related Contents:

  1. How to remove \xa0 from string in Python?
  2. Read and Write CSV files including unicode with Python 2.7
  3. Python – ‘ascii’ codec can’t decode byte
  4. How to convert a string to utf-8 in Python
  5. Get unicode code point of a character using Python
  6. Why does ENcoding a string result in a DEcoding error (UnicodeDecodeError)?
  7. How can I use io.StringIO() with the csv module?
  8. Python returns length of 2 for single Unicode character string
  9. How to sort a dataFrame in python pandas by two or more columns?
  10. How to get string objects instead of Unicode from JSON?
  11. Why isn’t my Pandas ‘apply’ function referencing multiple columns working? [closed]
  12. Why does Python print unicode characters when the default encoding is ASCII?
  13. Split a python list into other “sublists” i.e smaller lists [duplicate]
  14. Running an Excel macro via Python?
  15. What is the difference between encode/decode?
  16. How do I install the yaml package for Python?
  17. Python 3 print() function with Farsi/Arabic characters [duplicate]
  18. Search for a value in a nested dictionary python
  19. How can I do a batch insert into an Oracle database using Python?
  20. Python – Unicode to ASCII conversion
  21. Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence
  22. How to convert integer into date object python?
  23. Trying to use open(filename, ‘w’ ) gives IOError: [Errno 2] No such file or directory if directory doesn’t exist
  24. Printing a string prints ‘u’ before the string in Python?
  25. What are the differences between bool() and operator.truth()?
  26. how to open chrome in incognito mode from Python
  27. Matplotlib just giving error messages
  28. Understanding Pickling in Python
  29. How can I read pdf in python? [duplicate]
  30. Searching in Google with Python

Leave a Comment