Python not able to open file with non-english characters in path

by

The path in your error is:

'\xe3\x82\xaf\xe3\x83\xac\xe3\x82\xa4\xe3\x82\xb8\xe3\x83\xbc\xe3\x83\xbb\xe3\x83\x92\xe3\x83\x83\xe3\x83\x84\xef\xbc\x81'

I think this is the UTF8 encoded version of your filename.

I’ve created a folder of the same name on Windows7 and placed a file called ‘abc.txt’ in it:

>>> a="\xe3\x82\xaf\xe3\x83\xac\xe3\x82\xa4\xe3\x82\xb8\xe3\x83\xbc\xe3\x83\xbb\xe3\x83\x92\xe3\x83\x83\xe3\x83\x84\xef\xbc\x81"
>>> os.listdir('.')
['?????\xb7???!']
>>> os.listdir(u'.') # Pass unicode to have unicode returned to you
[u'\u30af\u30ec\u30a4\u30b8\u30fc\u30fb\u30d2\u30c3\u30c4\uff01']
>>> 
>>> a.decode('utf8') # UTF8 decoding your string matches the listdir output
u'\u30af\u30ec\u30a4\u30b8\u30fc\u30fb\u30d2\u30c3\u30c4\uff01'
>>> os.listdir(a.decode('utf8'))
[u'abc.txt']

So it seems that Duncan’s suggestion of path.decode('utf8') does the trick.

Update

I can’t test this for you, but I suggest that you try checking whether the path contains non-ascii before doing the .decode('utf8'). This is a bit hacky…

ASCII_TRANS = '_'*32 + ''.join([chr(x) for x in range(32,126)]) + '_'*130
path=path.strip()
path=path[17:] #to remove the file://localhost/  part
path=urllib.unquote(path)
if path.translate(ASCII_TRANS) != path: # Contains non-ascii
  path = path.decode('utf8')
path=urllib.url2pathname(path)

More Related Contents:

  1. Adding Python to PATH on Windows
  2. Writing a Python list of lists to a csv file
  3. Reading a huge .csv file
  4. Why am I getting a FileNotFoundError?
  5. Open file in a relative location in Python
  6. How to get rid of double backslash in python windows file path string? [duplicate]
  7. file.tell() inconsistency
  8. Python list directory, subdirectory, and files
  9. Fastest way to process a large file?
  10. Python 2.7: Print to File
  11. Is there a need to close files that have no reference to them?
  12. Python: import module from another directory at the same level in project hierarchy
  13. Mixing files and loops
  14. Sending a file over TCP sockets in Python
  15. How to read the last line of a file in Python? [duplicate]
  16. How can I extract the folder path from file path in Python?
  17. How can I import files in Python using sys.path.append?
  18. Cannot find the file specified when using subprocess.call(‘dir’, shell=True) in Python
  19. Searching for IP addresses in a file
  20. Swapping variable places inside if throws an error
  21. Export a list of stock prices to excel
  22. Text parsing and retrieval
  23. Do I understand os.walk right?
  24. In Python, why is list[] automatically global?
  25. Unable to install Python packages using pip in Ubuntu Linux: InsecurePlatformWarning, SSLError, tlsv1 alert protocol version
  26. Creating a new file, filename contains loop variable, python [duplicate]
  27. Spell Checker for Python
  28. Enforcing python version in setup.py
  29. subprocess.Popen in different console
  30. how to count the total number of lines in a text file using python

Leave a Comment