Non blocking subprocess.call

by

You should use subprocess.Popen instead of subprocess.call.

Something like:

subprocess.Popen(["python", "slave.py"] + sys.argv[1:])

From the docs on subprocess.call:

Run the command described by args. Wait for command to complete, then return the returncode attribute.

(Also don’t use a list to pass in the arguments if you’re going to use shell = True).

Here’s a MCVE1 example that demonstrates a non-blocking suprocess call:

import subprocess
import time

p = subprocess.Popen(['sleep', '5'])

while p.poll() is None:
    print('Still sleeping')
    time.sleep(1)

print('Not sleeping any longer.  Exited with returncode %d' % p.returncode)

An alternative approach that relies on more recent changes to the python language to allow for co-routine based parallelism is:

# python3.5 required but could be modified to work with python3.4.
import asyncio

async def do_subprocess():
    print('Subprocess sleeping')
    proc = await asyncio.create_subprocess_exec('sleep', '5')
    returncode = await proc.wait()
    print('Subprocess done sleeping.  Return code = %d' % returncode)

async def sleep_report(number):
    for i in range(number + 1):
        print('Slept for %d seconds' % i)
        await asyncio.sleep(1)

loop = asyncio.get_event_loop()

tasks = [
    asyncio.ensure_future(do_subprocess()),
    asyncio.ensure_future(sleep_report(5)),
]

loop.run_until_complete(asyncio.gather(*tasks))
loop.close()

1Tested on OS-X using python2.7 & python3.6

More Related Contents:

  1. Actual meaning of ‘shell=True’ in subprocess
  2. How to terminate a python subprocess launched with shell=True
  3. Python subprocess readlines() hangs
  4. How to redirect output with subprocess in Python?
  5. Python subprocess get children’s output to file and terminal?
  6. Subprocess changing directory
  7. Run a program from python, and have it continue to run after the script is killed
  8. Popen waiting for child process even when the immediate child has terminated
  9. Understanding Popen.communicate
  10. Execute terminal command from python in new terminal window?
  11. Python: how to kill child process(es) when parent dies?
  12. OSError: [WinError 193] %1 is not a valid Win32 application
  13. subprocess.Popen() error (No such file or directory) when calling command with arguments as a string
  14. Using subprocess to run Python script on Windows
  15. running multiple bash commands with subprocess
  16. Keep a subprocess alive and keep giving it commands? Python
  17. Python output to Console within Subprocess from the child scricpt
  18. python getoutput() equivalent in subprocess [duplicate]
  19. Why subprocess.Popen doesn’t work when args is sequence?
  20. How to run multiple commands synchronously from one subprocess.Popen command?
  21. Redirect subprocess stderr to stdout
  22. Interacting with bash from python
  23. Run BASH built-in commands in Python?
  24. Event Driven System Call in Python
  25. How do I eliminate Windows consoles from spawned processes in Python (2.7)? [duplicate]
  26. Python Subprocess: Too Many Open Files
  27. How can I spawn new shells to run Python scripts from a base Python script?
  28. Handling directories with spaces Python subprocess.call()
  29. Opening a process with Popen and getting the PID
  30. Subprocess check_output returned non-zero exit status 1

Leave a Comment