Blocking and Non Blocking subprocess calls

by

Popen is nonblocking. call and check_call are blocking.
You can make the Popen instance block by calling its wait or communicate method.

If you look in the source code, you’ll see call calls Popen(...).wait(), which is why it is blocking.
check_call calls call, which is why it blocks as well.

Strictly speaking, shell=True is orthogonal to the issue of blocking. However, shell=True causes Python to exec a shell and then run the command in the shell. If you use a blocking call, the call will return when the shell finishes. Since the shell may spawn a subprocess to run the command, the shell may finish before the spawned subprocess. For example,

import subprocess
import time

proc = subprocess.Popen('ls -lRa /', shell=True)
time.sleep(3)
proc.terminate()
proc.wait()

Here two processes are spawned: Popen spawns one subprocess running the shell. The shell in turn spawns a subprocess running ls. proc.terminate() kills the shell, but the subprocess running ls remains. (That is manifested by copious output, even after the python script has ended. Be prepared to kill the ls with pkill ls.)

More Related Contents:

  1. Actual meaning of ‘shell=True’ in subprocess
  2. How to terminate a python subprocess launched with shell=True
  3. How to use `subprocess` command with pipes
  4. Python subprocess readlines() hangs
  5. How to redirect output with subprocess in Python?
  6. Python subprocess get children’s output to file and terminal?
  7. Subprocess changing directory
  8. subprocess.call using string vs using list
  9. Understanding Popen.communicate
  10. Execute terminal command from python in new terminal window?
  11. Python: how to kill child process(es) when parent dies?
  12. OSError: [WinError 193] %1 is not a valid Win32 application
  13. subprocess.Popen() error (No such file or directory) when calling command with arguments as a string
  14. Using subprocess to run Python script on Windows
  15. running multiple bash commands with subprocess
  16. Keep a subprocess alive and keep giving it commands? Python
  17. Python output to Console within Subprocess from the child scricpt
  18. python getoutput() equivalent in subprocess [duplicate]
  19. Why subprocess.Popen doesn’t work when args is sequence?
  20. How to run multiple commands synchronously from one subprocess.Popen command?
  21. Redirect subprocess stderr to stdout
  22. Interacting with bash from python
  23. Event Driven System Call in Python
  24. How do I eliminate Windows consoles from spawned processes in Python (2.7)? [duplicate]
  25. Launch IPython notebook with selected browser
  26. Sending ^C to Python subprocess objects on Windows
  27. Execute Commands Sequentially in Python?
  28. Killing child process when parent crashes in python
  29. How to install and import Python modules at runtime
  30. How to terminate long-running computation (CPU bound task) in Python using asyncio and concurrent.futures.ProcessPoolExecutor?

Leave a Comment