How to unquote a urlencoded unicode string in python?

by

%uXXXX is a non-standard encoding scheme that has been rejected by the w3c, despite the fact that an implementation continues to live on in JavaScript land.

The more common technique seems to be to UTF-8 encode the string and then % escape the resulting bytes using %XX. This scheme is supported by urllib.unquote:

>>> urllib2.unquote("%0a")
'\n'

Unfortunately, if you really need to support %uXXXX, you will probably have to roll your own decoder. Otherwise, it is likely to be far more preferable to simply UTF-8 encode your unicode and then % escape the resulting bytes.

A more complete example:

>>> u"Tanım"
u'Tan\u0131m'
>>> url = urllib.quote(u"Tanım".encode('utf8'))
>>> urllib.unquote(url).decode('utf8')
u'Tan\u0131m'

More Related Contents:

  1. Writing Unicode text to a text file?
  2. Convert Unicode to ASCII without errors in Python
  3. Get a list of all the encodings Python can encode to
  4. What is the difference between encode/decode?
  5. How to download any(!) webpage with correct charset in python?
  6. Bytes in a unicode Python string
  7. Python, Unicode, and the Windows console
  8. Downloading a picture via urllib and python
  9. can we use XPath with BeautifulSoup?
  10. SyntaxError: Non-ASCII character ‘\xa3’ in file when function returns ‘£’
  11. What’s the deal with Python 3.4, Unicode, different languages and Windows?
  12. AttributeError: ‘module’ object has no attribute ‘urlopen’
  13. How to fetch a non-ascii url with urlopen?
  14. Where is Python’s “best ASCII for this Unicode” database? [closed]
  15. How to convert a string to utf-8 in Python
  16. Python unicode equal comparison failed
  17. General Unicode/UTF-8 support for csv files in Python 2.6
  18. How to correctly parse UTF-8 encoded HTML to Unicode strings with BeautifulSoup? [duplicate]
  19. Python: Get HTTP headers from urllib2.urlopen call?
  20. Python – can I detect unicode string language code?
  21. How to make unicode string with python3
  22. Get unicode code point of a character using Python
  23. How to replace unicode characters in string with something else python?
  24. Get size of a file before downloading in Python
  25. How can I check if a Python unicode string contains non-Western letters?
  26. How do convert unicode escape sequences to unicode characters in a python string
  27. Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str
  28. requests.get returns 403 while the same url works in browser
  29. What is internal representation of string in Python 3.x
  30. How to send non-English unicode string using HTTP header?

Leave a Comment