Python subprocess .check_call vs .check_output

by

check_call() returns as soon as /bin/sh process exits without waiting for descendant processes (assuming shell=True as in your case).

check_output() waits until all output is read. If ssh inherits the pipe then check_output() will wait until it exits (until it closes its inherited pipe ends).

check_call() code example:

#!/usr/bin/env python
import subprocess
import sys
import time

start = time.time()
cmd = sys.executable + " -c 'import time; time.sleep(2)' &"
subprocess.check_call(cmd, shell=True)
assert (time.time() - start) < 1

The output is not read; check_call() returns immediately without waiting for the grandchild background python process.

check_call() is just Popen().wait(). Popen() starts the external process and returns immediately without waiting for it to exit. .wait() collects the exit status for the process — it doesn’t wait for other (grandchildren) processes.

If the output is read (it is redirected and the grandchild python
process inherits the stdout pipe):

start = time.time()
subprocess.check_output(cmd, shell=True)
assert (time.time() - start) > 2

then it waits until the background python process that inherited the pipe exits.

check_output() calls Popen().communicate(), to get the output. .communicate() calls .wait() internally i.e., check_output() also waits for the shell to exit and check_output() waits for EOF.

If the grandchild doesn’t inherit the pipe then check_output() doesn’t wait for it:

start = time.time()
cmd = sys.executable + " -c 'import time; time.sleep(2)' >/dev/null &"
subprocess.check_output(cmd, shell=True)
assert (time.time() - start) < 1

Grandchild’s output is redirected to /dev/null i.e., it doesn’t inherit the parent’s pipe and therefore check_output() may exit without waiting for it.

Note: & at the end which puts the grandchild python process into background. It won’t work on Windows where shell=True starts cmd.exe by default.

More Related Contents:

  1. Pass input/variables to command/script over SSH using Python Paramiko
  2. Python subprocess.call a bash alias
  3. subprocess.call() arguments ignored when using shell=True w/ list [duplicate]
  4. running multiple bash commands with subprocess
  5. Process substitution not allowed by Python’s subprocess with shell=True?
  6. Python output to Console within Subprocess from the child scricpt
  7. Interacting with bash from python
  8. Run BASH built-in commands in Python?
  9. Sending a password over SSH or SCP with subprocess.Popen
  10. How to catch exception output from Python subprocess.check_output()?
  11. Launch a python script from another script, with parameters in subprocess argument
  12. Bash style process substitution with Python’s Popen
  13. Execute Subprocess in Background
  14. A non-blocking read on a subprocess.PIPE in Python
  15. How to hide output of subprocess
  16. Retrieving the output of subprocess.call() [duplicate]
  17. Constantly print Subprocess output while process is running
  18. How to scp in Python?
  19. Python – Activate conda env through shell script
  20. Reading file opened with Python Paramiko SFTPClient.open method is slow
  21. How to implement common bash idioms in Python? [closed]
  22. Ensuring subprocesses are dead on exiting Python program
  23. Execute command and wait for it to finish with Python Paramiko
  24. How to run sudo with Paramiko? (Python)
  25. Passing variables to a subprocess call [duplicate]
  26. Why does subprocess.Popen() with shell=True work differently on Linux vs Windows?
  27. Pipe input to script and later get input from user
  28. Python Subprocess: Too Many Open Files
  29. Handling directories with spaces Python subprocess.call()
  30. Opening a process with Popen and getting the PID

Leave a Comment