Multiple inputs and outputs in python subprocess communicate

by

Popen.communicate() is a helper method that does a one-time write of data to stdin and creates threads to pull data from stdout and stderr. It closes stdin when its done writing data and reads stdout and stderr until those pipes close. You can’t do a second communicate because the child has already exited by the time it returns.

An interactive session with a child process is quite a bit more complicated.

One problem is whether the child process even recognizes that it should be interactive. In the C libraries that most command line programs use for interaction, programs run from terminals (e.g., a linux console or “pty” pseudo-terminal) are interactive and flush their output frequently, but those run from other programs via PIPES are non-interactive and flush their output infrequently.

Another is how you should read and process stdout and stderr without deadlocking. For instance, if you block reading stdout, but stderr fills its pipe, the child will halt and you are stuck. You can use threads to pull both into internal buffers.

Yet another is how you deal with a child that exits unexpectedly.

For “unixy” systems like linux and OSX, the pexpect module is written to handle the complexities of an interactive child process. For Windows, there is no good tool that I know of to do it.

More Related Contents:

  1. Interactive input/output using Python
  2. Running an interactive command from within Python
  3. How do I pass a string into subprocess.Popen (using the stdin argument)?
  4. How do I write to a Python subprocess’ stdin?
  5. How to open every file in a folder
  6. catching stdout in realtime from subprocess
  7. Displaying subprocess output to stdout and redirecting it
  8. Run command and get its stdout, stderr separately in near real time like in a terminal
  9. Intercepting stdout of a subprocess while it is running
  10. subprocess readline hangs waiting for EOF
  11. Python read from subprocess stdout and stderr separately while preserving order
  12. How do I get ‘real-time’ information back from a subprocess.Popen in python (2.5)
  13. How do I check if stdin has some data?
  14. How to replicate tee behavior in Python when using subprocess?
  15. Piping output of subprocess.Popen to files
  16. “subprocess.Popen” – checking for success and errors
  17. How can I capture the stdout output of a child process?
  18. Python subprocess output to stdout
  19. Running shell command and capturing the output
  20. subprocess wildcard usage
  21. Non blocking subprocess.call
  22. How do I read from stdin?
  23. Use subprocess to send a password
  24. When to use Shell=True for Python subprocess module [duplicate]
  25. ‘^’ is ignored by Python – how to escape ‘^’ character in Popen Windows?
  26. Break the function after certain time
  27. Python subprocess in parallel
  28. Pipe input to script and later get input from user
  29. Run subprocess and print output to logging
  30. subprocess call ffmpeg (command line)

Leave a Comment