Unicode filenames on Windows with Python & subprocess.Popen()

by

In Py3K – at least from Python 3.2 – subprocess.Popen and sys.argv work consistently with (default unicode) strings on Windows. CreateProcessW and GetCommandLineW are used obviously.

In Python – up to v2.7.2 at least – subprocess.Popen is buggy with Unicode arguments. It sticks to CreateProcessA (while os.* are consistent with Unicode). And shlex.split creates additional nonsense.

Pywin32’s win32process.CreateProcess also doesn’t auto-switch to the W version, nor is there a win32process.CreateProcessW. Same with GetCommandLine.
Thus ctypes.windll.kernel32.CreateProcessW... needs to be used.
The subprocess module perhaps should be fixed regarding this issue.

UTF8 on argv[1:] with private apps remains clumsy on a Unicode OS. Such tricks may be legal for 8-bit “Latin1” string OSes like Linux.

UPDATE vaab has created a patched version of Popen for Python 2.7 which fixes the issue.
See https://gist.github.com/vaab/2ad7051fc193167f15f85ef573e54eb9
Blog post with explanations: http://vaab.blog.kal.fr/2017/03/16/fixing-windows-python-2-7-unicode-issue-with-subprocesss-popen/

More Related Contents:

  1. Windows cmd encoding change causes Python crash
  2. Read Unicode characters from command-line arguments in Python 2.x on Windows
  3. python: unicode in Windows terminal, encoding used?
  4. chcp 65001 codepage results in program termination without any error
  5. What exactly do “u” and “r” string flags do, and what are raw string literals?
  6. pip install fails with “connection error: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:598)”
  7. How do you fix “runtimeError: package fails to pass a sanity check” for numpy and pandas?
  8. Executing periodic actions [duplicate]
  9. “Cannot open include file: ‘config-win.h’: No such file or directory” while installing mysql-python
  10. Get screenshot on Windows with Python?
  11. PyWin32 and Python 3.8.0
  12. Can I install Python 3.x and 2.x on the same Windows computer?
  13. urllib2 read to Unicode
  14. ‘UCS-2’ codec can’t encode characters in position 1050-1050
  15. Why doesn’t Python recognize my utf-8 encoded source file?
  16. How to make Django slugify work properly with Unicode strings?
  17. Removing minimize/maximize buttons in Tkinter
  18. Handling \r\n vs \n newlines in python on Mac vs Windows
  19. Unicode identifiers in Python?
  20. Running Python Script as a Windows background process [duplicate]
  21. Using Windows Python from Cygwin
  22. How does unicodedata.normalize(form, unistr) work?
  23. Python – dealing with mixed-encoding files
  24. Python the same char not equals
  25. Trying to create a virtualenv in python and activate it
  26. Error: No module named ‘fcntl’
  27. Python Unicode string stored as ‘\u84b8\u6c7d\u5730’ in file, how to convert it back to Unicode?
  28. ‘pip setup.py bdist_wheel’ no longer builds forced non-pure wheels
  29. Add “django-admin.py” path to command line on Windows 7
  30. Python long filename support broken in Windows

Leave a Comment