UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xef in position 1

by

This is to do with the encoding of your terminal not being set to UTF-8. Here is my terminal

$ echo $LANG
en_GB.UTF-8
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s="(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89"
>>> s1 = s.decode('utf-8')
>>> print s1
(｡･ω･｡)ﾉ
>>>

On my terminal the example works with the above, but if I get rid of the LANG setting then it won’t work

$ unset LANG
$ python
Python 2.7.3 (default, Apr 20 2012, 22:39:59) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> s="(\xef\xbd\xa1\xef\xbd\xa5\xcf\x89\xef\xbd\xa5\xef\xbd\xa1)\xef\xbe\x89"
>>> s1 = s.decode('utf-8')
>>> print s1
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-5: ordinal not in range(128)
>>>

Consult the docs for your linux variant to discover how to make this change permanent.

More Related Contents:

  1. Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence
  2. How to remove \xa0 from string in Python?
  3. u’\ufeff’ in Python string
  4. Unicode (UTF-8) reading and writing to files in Python
  5. Convert Unicode to ASCII without errors in Python
  6. Python & MySql: Unicode and Encoding
  7. Python and BeautifulSoup encoding issues [duplicate]
  8. How to convert a string to utf-8 in Python
  9. How do I check if a string is unicode or ascii?
  10. Writing UTF-8 String to MySQL with Python
  11. General Unicode/UTF-8 support for csv files in Python 2.6
  12. MySQL “incorrect string value” error when save unicode string in Django
  13. Why doesn’t Python recognize my utf-8 encoded source file?
  14. How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? [duplicate]
  15. Python 3 print() function with Farsi/Arabic characters [duplicate]
  16. Python ascii utf unicode
  17. Python DictWriter writing UTF-8 encoded CSV files
  18. ElementTree and unicode
  19. Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence
  20. Convert UTF-16 to UTF-8 and remove BOM?
  21. Convert “\x” escaped string into readable string in python
  22. python encoding utf-8
  23. Python – dealing with mixed-encoding files
  24. SQLite, python, unicode, and non-utf data
  25. UnicodeDecodeError when performing os.walk
  26. Bytes in a unicode Python string
  27. Python – Unicode to ASCII conversion
  28. What does ‘u’ mean in a list?
  29. Python the same char not equals
  30. How to send non-English unicode string using HTTP header?

Leave a Comment