How do I get str.translate to work with Unicode strings?

by

The Unicode version of translate requires a mapping from Unicode ordinals (which you can retrieve for a single character with ord) to Unicode ordinals. If you want to delete characters, you map to None.

I changed your function to build a dict mapping the ordinal of every character to the ordinal of what you want to translate to:

def translate_non_alphanumerics(to_translate, translate_to=u'_'):
    not_letters_or_digits = u'!"#%\'()*+,-./:;<=>?@[\]^_`{|}~'
    translate_table = dict((ord(char), translate_to) for char in not_letters_or_digits)
    return to_translate.translate(translate_table)

>>> translate_non_alphanumerics(u'<foo>!')
u'_foo__'

edit: It turns out that the translation mapping must map from the Unicode ordinal (via ord) to either another Unicode ordinal, a Unicode string, or None (to delete). I have thus changed the default value for translate_to to be a Unicode literal. For example:

>>> translate_non_alphanumerics(u'<foo>!', u'bad')
u'badfoobadbad'

More Related Contents:

  1. What does the ‘b’ character do in front of a string literal?
  2. Convert a Unicode string to a string in Python (containing extra symbols)
  3. removing emojis from a string in Python
  4. How to check if a string in Python is in ASCII?
  5. Suppress the u’prefix indicating unicode’ in python strings
  6. Python str vs unicode types
  7. Python __str__ versus __unicode__
  8. What is the difference between encode/decode?
  9. byte string vs. unicode string. Python
  10. What does ‘u’ mean in a list?
  11. How is unicode represented internally in Python?
  12. How to detect string byte encoding?
  13. Python string to unicode [duplicate]
  14. What is internal representation of string in Python 3.x
  15. Best way to strip punctuation from a string
  16. How to check if a string is a substring of items in a list of strings?
  17. How to get a function name as a string?
  18. Python UnicodeDecodeError – Am I misunderstanding encode?
  19. How to add a string in a certain position?
  20. How to remove square bracket from pandas dataframe
  21. Converting a string that represents a list, into an actual list object
  22. How to search if dictionary value contains certain string with Python
  23. How to concatenate (join) items in a list to a single string
  24. python: unicode in Windows terminal, encoding used?
  25. What is a Python bytestring?
  26. Remove whitespace in Python using string.whitespace
  27. Slicing strings in str.format
  28. string.translate() with unicode data in python
  29. Python split string in moving window
  30. Convert bytes to a string in python 3

Leave a Comment