UnicodeEncodeError: ‘ascii’ codec can’t encode characters in position 0-5: ordinal not in range(128) [duplicate]

by

Python is trying to be helpful. You cannot decode Unicode data, it is already decoded. So Python first will encode the data (using the ASCII codec) to get bytes to decode. It is this implicit encoding that fails.

If you have Unicode data, it only makes sense to encode to UTF-8, not decode:

>>> print u'\u041e\u043b\u044c\u0433\u0430'
Ольга
>>> u'\u041e\u043b\u044c\u0433\u0430'.encode('utf8')
'\xd0\x9e\xd0\xbb\xd1\x8c\xd0\xb3\xd0\xb0'

If you wanted a Unicode value, then using a Unicode literal (u'...') is all you needed to do. No further decoding is necessary.

The same implicit conversion takes place in the other direction; if you tried to encode a bytestring you’d trigger an implicit decoding:

>>> u'\u041e\u043b\u044c\u0433\u0430'.encode('utf8').encode('utf8')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd0 in position 0: ordinal not in range(128)

More Related Contents:

  1. How to remove \xa0 from string in Python?
  2. How to convert a string to utf-8 in Python
  3. Python reading from a file and saving to utf-8
  4. Oops, try again. Your function failed on the message yes. It returned 'yes' when it should have returned 'Shutting down'
  5. Python returns "SyntaxError: invalid syntax sys module" [closed]
  6. Why are Python’s ‘private’ methods not actually private?
  7. Python 2.7 getting user input and manipulating as string without quotations
  8. UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xef in position 1
  9. Unable to locate element of credit card number using selenium python
  10. How to convert a given ordinal number (from Excel) to a date
  11. UnicodeDecodeError, invalid continuation byte
  12. How would I stop a while loop after n amount of time?
  13. How to extract text from an existing docx file using python-docx
  14. ImportError: No module named _ssl
  15. Python Save to file
  16. Unnamed Python objects have the same id
  17. Redirecting stdout to “nothing” in python
  18. How can PySpark be called in debug mode?
  19. Assigning string with boolean expression [duplicate]
  20. Why can I pass an instance method to multiprocessing.Process, but not a multiprocessing.Pool?
  21. How to get Windows short file name in python?
  22. How to plot a chart in the terminal
  23. Why does this Python program send empty emails when I encode it with utf-8? [duplicate]
  24. Behaviour of raw_input()
  25. Byte Array to Hex String
  26. Sort a list with a custom order in Python
  27. Convert “\x” escaped string into readable string in python
  28. Kivy error (python 2.7) : sdl2 import error
  29. Get exception description and stack trace which caused an exception, all as a string
  30. What does version name ‘cp27’ or ‘cp35’ mean in Python?

Leave a Comment