Writing UTF-8 String to MySQL with Python

by

As @marr75 suggests, make sure you set charset="utf8" on your connections. Setting use_unicode=True is not strictly necessary as it is implied by setting the charset.

Then make sure you are passing unicode objects to your db connection as it will encode it using the charset you passed to the cursor. If you are passing a utf8-encoded string, it will be doubly encoded when it reaches the database.

So, something like:

conn = MySQLdb.connect(host="localhost", user="root", password='', db='', charset="utf8")
data_from_ldap = 'M\xc3\xbcller'
name = data_from_ldap.decode('utf8')
cursor = conn.cursor()
cursor.execute(u"INSERT INTO mytable SET name = %s", (name,))

You may also try forcing the connection to use utf8 by passing the init_command param, though I’m unsure if this is required. 5 mins testing should help you decide.

conn = MySQLdb.connect(charset="utf8", init_command='SET NAMES UTF8')

Also, and this is barely worth mentioning as 4.1 is so old, make sure you are using MySQL >= 4.1

More Related Contents:

  1. Saving utf-8 texts with json.dumps as UTF8, not as \u escape sequence
  2. How to remove \xa0 from string in Python?
  3. u’\ufeff’ in Python string
  4. Unicode (UTF-8) reading and writing to files in Python
  5. UnicodeDecodeError: ‘ascii’ codec can’t decode byte 0xef in position 1
  6. Convert Unicode to ASCII without errors in Python
  7. Python & MySql: Unicode and Encoding
  8. Python and BeautifulSoup encoding issues [duplicate]
  9. How to convert a string to utf-8 in Python
  10. How do I check if a string is unicode or ascii?
  11. General Unicode/UTF-8 support for csv files in Python 2.6
  12. MySQL “incorrect string value” error when save unicode string in Django
  13. Why doesn’t Python recognize my utf-8 encoded source file?
  14. How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? [duplicate]
  15. Python 3 print() function with Farsi/Arabic characters [duplicate]
  16. Python ascii utf unicode
  17. Python DictWriter writing UTF-8 encoded CSV files
  18. ElementTree and unicode
  19. Saving UTF-8 texts with json.dumps as UTF-8, not as a \u escape sequence
  20. Convert UTF-16 to UTF-8 and remove BOM?
  21. Convert “\x” escaped string into readable string in python
  22. python encoding utf-8
  23. Python – dealing with mixed-encoding files
  24. SQLite, python, unicode, and non-utf data
  25. UnicodeDecodeError when performing os.walk
  26. Bytes in a unicode Python string
  27. Windows cmd encoding change causes Python crash
  28. How to resolve TypeError: can only concatenate str (not “int”) to str [duplicate]
  29. python requests.get() returns improperly decoded text instead of UTF-8?
  30. How to send non-English unicode string using HTTP header?

Leave a Comment