unicode().decode(‘utf-8’, ‘ignore’) raising UnicodeEncodeError

by

When I first started messing around with python strings and unicode, It took me awhile to understand the jargon of decode and encode too, so here’s my post from here that may help:

Think of decoding as what you do to go from a regular bytestring to unicode and encoding as what you do to get back from unicode. In other words:

You de-code a str to produce a unicode string (in Python 2)

and en-code a unicode string to produce a str (in Python 2)

So:

unicode_char = u'\xb0'

encodedchar = unicode_char.encode('utf-8')

encodedchar will contain your unicode character, displayed in the selected encoding (in this case, utf-8).

The same principle applies to Python 3. You de-code a bytes object to produce a str object. And you en-code a str object to produce a bytes object.

More Related Contents:

  1. What is the best way to remove accents (normalize) in a Python unicode string?
  2. What’s the complete range for Chinese characters in Unicode?
  3. FPDF utf-8 encoding (HOW-TO)
  4. Python – ‘ascii’ codec can’t decode byte
  5. Is there a Windows command shell that will display Unicode characters?
  6. General Unicode/UTF-8 support for csv files in Python 2.6
  7. What’s the best option to display Unicode text (hebrew, etc.) in VB6
  8. Unicode range for Japanese
  9. Using redirection within the script produces a unicode output. How to emit single-byte ASCII text?
  10. Python ascii utf unicode
  11. How can I clean source code files of invisible characters?
  12. Can UTF-8 contain zero byte?
  13. List of all unicode’s open/close brackets?
  14. grepping binary files and UTF16
  15. How do convert unicode escape sequences to unicode characters in a python string
  16. What’s the difference between UTF-8 and UTF-8 with BOM?
  17. Read Unicode characters from command-line arguments in Python 2.x on Windows
  18. Printing Unicode characters to the PowerShell prompt
  19. Difference between Big Endian and little Endian Byte order
  20. What is the range of Unicode Printable Characters?
  21. Print Directory & File Structure with icons for representation in Markdown [closed]
  22. Python string to unicode [duplicate]
  23. How to read utf16 text file to string in golang?
  24. Classic ASP: How to write unicode string data in classic ASP?
  25. What does it mean when my text is displayed as Question Marks?
  26. ASCII vs Unicode + UTF-8
  27. Should I support Unicode in passwords?
  28. UTF-8 Continuation bytes
  29. Unicode output on Windows command line?
  30. Why is Unicode restricted to 0x10FFFF?

Leave a Comment