u’\ufeff’ in Python string

by

I ran into this on Python 3 and found this question (and solution).
When opening a file, Python 3 supports the encoding keyword to automatically handle the encoding.

Without it, the BOM is included in the read result:

>>> f = open('file', mode="r")
>>> f.read()
'\ufefftest'

Giving the correct encoding, the BOM is omitted in the result:

>>> f = open('file', mode="r", encoding='utf-8-sig')
>>> f.read()
'test'

Just my 2 cents.

More Related Contents:

  1. Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence
  2. How to remove \xa0 from string in Python?
  3. Unicode (UTF-8) reading and writing to files in Python
  4. UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xef in position 1
  5. Convert Unicode to ASCII without errors in Python
  6. Python & MySql: Unicode and Encoding
  7. Python and BeautifulSoup encoding issues [duplicate]
  8. How to convert a string to utf-8 in Python
  9. How do I check if a string is unicode or ascii?
  10. Writing UTF-8 String to MySQL with Python
  11. General Unicode/UTF-8 support for csv files in Python 2.6
  12. MySQL “incorrect string value” error when save unicode string in Django
  13. Why doesn’t Python recognize my utf-8 encoded source file?
  14. How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? [duplicate]
  15. Python 3 print() function with Farsi/Arabic characters [duplicate]
  16. Python ascii utf unicode
  17. Python DictWriter writing UTF-8 encoded CSV files
  18. ElementTree and unicode
  19. Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence
  20. Convert UTF-16 to UTF-8 and remove BOM?
  21. Convert “\x” escaped string into readable string in python
  22. python encoding utf-8
  23. Python – dealing with mixed-encoding files
  24. SQLite, python, unicode, and non-utf data
  25. UnicodeDecodeError when performing os.walk
  26. Bytes in a unicode Python string
  27. remove unicode emoji using re in python
  28. latin-1 to ascii
  29. Python 3 UnicodeDecodeError: ‘charmap’ codec can’t decode byte 0x9d
  30. Remove non-ASCII characters from a string using python / django

Leave a Comment