Python Unicode Encode Error

by

Likely, your problem is that you parsed it okay, and now you’re trying to print the contents of the XML and you can’t because theres some foreign Unicode characters. Try to encode your unicode string as ascii first:

unicodeData.encode('ascii', 'ignore')

the ‘ignore’ part will tell it to just skip those characters. From the python docs:

>>> # Python 2: u = unichr(40960) + u'abcd' + unichr(1972)
>>> u = chr(40960) + u'abcd' + chr(1972)
>>> u.encode('utf-8')
'\xea\x80\x80abcd\xde\xb4'
>>> u.encode('ascii')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character '\ua000' in position 0: ordinal not in range(128)
>>> u.encode('ascii', 'ignore')
'abcd'
>>> u.encode('ascii', 'replace')
'?abcd?'
>>> u.encode('ascii', 'xmlcharrefreplace')
'&#40960;abcd&#1972;'

You might want to read this article: http://www.joelonsoftware.com/articles/Unicode.html, which I found very useful as a basic tutorial on what’s going on. After the read, you’ll stop feeling like you’re just guessing what commands to use (or at least that happened to me).

More Related Contents:

  1. Python UnicodeDecodeError – Am I misunderstanding encode?
  2. Replace non-ASCII characters with a single space
  3. How can I remove non-ASCII characters but leave periods and spaces?
  4. How to check if a string in Python is in ASCII?
  5. Convert Unicode to ASCII without errors in Python
  6. Python string prints as [u’String’]
  7. Why does Python print unicode characters when the default encoding is ASCII?
  8. Character reading from file in Python
  9. Python & MySql: Unicode and Encoding
  10. Where is Python’s “best ASCII for this Unicode” database? [closed]
  11. Non-ASCII characters in Matplotlib
  12. UnicodeEncodeError: ‘ascii’ codec can’t encode character at special name [duplicate]
  13. Python – Unicode to ASCII conversion
  14. How do I treat an ASCII string as unicode and unescape the escaped characters in it in python?
  15. Convert int to ASCII and back in Python
  16. UnicodeEncodeError: ‘ascii’ codec can’t encode character u’\xa0′ in position 20: ordinal not in range(128)
  17. Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence
  18. How to get the ASCII value of a character
  19. u’\ufeff’ in Python string
  20. Writing Unicode text to a text file?
  21. removing emojis from a string in Python
  22. How can I convert surrogate pairs to normal string in Python?
  23. Windows cmd encoding change causes Python crash
  24. UnicodeDecodeError when redirecting to file
  25. Why does ENcoding a string result in a DEcoding error (UnicodeDecodeError)?
  26. “SyntaxError: Non-ASCII character …” or “SyntaxError: Non-UTF-8 code starting with …” trying to use non-ASCII text in a Python script
  27. Difference between open and codecs.open in Python
  28. python encoding utf-8
  29. Whitespace gone from PDF extraction, and strange word interpretation
  30. Python 3 chokes on CP-1252/ANSI reading

Leave a Comment