Python 3 UnicodeDecodeError: ‘charmap’ codec can’t decode byte 0x9d

by

In Python 3, files are opened as text (decoded to Unicode) for you; you don’t need to tell BeautifulSoup what codec to decode from.

If decoding of the data fails, that’s because you didn’t tell the open() call what codec to use when reading the file; add the correct codec with an encoding argument:

with open(filename, encoding='utf8') as infile:
    html = BeautifulSoup(infile, "html.parser")

otherwise the file will be opened with your system default codec, which is OS dependent.

More Related Contents:

  1. What is the best way to remove accents (normalize) in a Python unicode string?
  2. UnicodeDecodeError when reading CSV file in Pandas with Python
  3. How to set sys.stdout encoding in Python 3?
  4. Replace non-ASCII characters with a single space
  5. Python regex matching Unicode properties
  6. Convert Unicode to ASCII without errors in Python
  7. How to correct TypeError: Unicode-objects must be encoded before hashing?
  8. Python UnicodeDecodeError – Am I misunderstanding encode?
  9. Suppress the u’prefix indicating unicode’ in python strings
  10. How to make the python interpreter correctly handle non-ASCII characters in string operations?
  11. Character reading from file in Python
  12. Python & MySql: Unicode and Encoding
  13. Facebook JSON badly encoded
  14. Python __str__ versus __unicode__
  15. byte string vs. unicode string. Python
  16. How to unquote a urlencoded unicode string in python?
  17. How do I specify a range of unicode characters
  18. Python ascii utf unicode
  19. matching unicode characters in python regular expressions
  20. Unicode values in strings are escaped when dumping to JSON in Python [duplicate]
  21. How to filter (or replace) unicode characters that would take more than 3 bytes in UTF-8?
  22. Displaying unicode symbols using pygame
  23. Read Unicode characters from command-line arguments in Python 2.x on Windows
  24. python: unicode in Windows terminal, encoding used?
  25. How do I get str.translate to work with Unicode strings?
  26. Convert “\x” escaped string into readable string in python
  27. How to encode Python 3 string using \u escape code?
  28. Python returns length of 2 for single Unicode character string
  29. Set encoding in Python 3 CGI scripts
  30. Python 3 chokes on CP-1252/ANSI reading

Leave a Comment