Python not sorting unicode properly. Strcoll doesn’t help

by

Apparently, the only way for sorting to work on all platforms is to use the ICU library with PyICU bindings (PyICU on PyPI).

On OS X: sudo port install py26-pyicu, minding bug described here: https://svn.macports.org/ticket/23429 (oh the joy of using macports).

PyICUs documentation is unfortunately severely lacking, but I managed to find out how it’s done:

import PyICU
collator = PyICU.Collator.createInstance(PyICU.Locale('pl_PL.UTF-8'))
print [i for i in sorted([u'a', u'z', u'ą'], cmp=collator.compare)]

which gives:

[u'a', u'ą', u'z']

Another pro – @bobince: it’s thread-safe, so not useless when setting request-wise locales.

More Related Contents:

  1. What exactly do “u” and “r” string flags do, and what are raw string literals?
  2. How to remove \xa0 from string in Python?
  3. How to get string objects instead of Unicode from JSON?
  4. Why does Python print unicode characters when the default encoding is ASCII?
  5. Python locale error: unsupported locale setting
  6. Get a list of all the encodings Python can encode to
  7. What is the difference between encode/decode?
  8. urllib2 read to Unicode
  9. ‘UCS-2’ codec can’t encode characters in position 1050-1050
  10. Why doesn’t Python recognize my utf-8 encoded source file?
  11. Python 3 print() function with Farsi/Arabic characters [duplicate]
  12. How to make Django slugify work properly with Unicode strings?
  13. Python 3: How to specify stdin encoding
  14. Python – Unicode to ASCII conversion
  15. Identifier normalization: Why is the micro sign converted into the Greek letter mu?
  16. How do I treat an ASCII string as unicode and unescape the escaped characters in it in python?
  17. python-re: How do I match an alpha character
  18. What does ‘u’ mean in a list?
  19. Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence
  20. Error “(unicode error) ‘unicodeescape’ codec can’t decode bytes in position 2-3: truncated \UXXXXXXXX escape” [duplicate]
  21. How is unicode represented internally in Python?
  22. Unicode identifiers in Python?
  23. Printing a string prints ‘u’ before the string in Python?
  24. How does unicodedata.normalize(form, unistr) work?
  25. Python – dealing with mixed-encoding files
  26. Python: How can I replace full-width characters with half-width characters?
  27. Python the same char not equals
  28. SQLite, python, unicode, and non-utf data
  29. Remove non-ASCII characters from a string using python / django
  30. Python Unicode string stored as ‘\u84b8\u6c7d\u5730’ in file, how to convert it back to Unicode?

Leave a Comment