Python Subprocess: Too Many Open Files

by

In Mac OSX (El Capitan) See current configuration:

#ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 256
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

Set open files value to 10K :

#ulimit -Sn 10000

Verify results:

#ulimit -a

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
file size               (blocks, -f) unlimited
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 10000
pipe size            (512 bytes, -p) 1
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 709
virtual memory          (kbytes, -v) unlimited

More Related Contents:

  1. Read streaming input from subprocess.communicate()
  2. How do I use subprocess.Popen to connect multiple processes by pipes?
  3. Passing Variables to Subprocess.Popen
  4. ImageMagick not authorized to convert PDF to an image
  5. How can I run an external command asynchronously from Python?
  6. catching stdout in realtime from subprocess
  7. How to get exit code when using Python subprocess communicate method?
  8. Displaying subprocess output to stdout and redirecting it
  9. Python subprocess.call a bash alias
  10. Pipe subprocess standard output to a variable [duplicate]
  11. Python read from subprocess stdout and stderr separately while preserving order
  12. How do I get all of the output from my .exe using subprocess and Popen?
  13. Is it possible to run function in a subprocess without threading or writing a separate file/script.
  14. What’s a good equivalent to subprocess.check_call that returns the contents of stdout?
  15. Cross-platform subprocess with hidden window
  16. Python read file as stream from HDFS
  17. Python subprocess: callback when cmd exits
  18. ‘yes’ reporting error with subprocess communicate()
  19. How do I pass large numpy arrays between python subprocesses without saving to disk?
  20. wait process until all subprocess finish? [duplicate]
  21. OSError: [Errno 8] Exec format error
  22. running a command as a super user from a python script
  23. Run multiple programs sequentially in one Windows command prompt?
  24. how to kill (or avoid) zombie processes with subprocess module
  25. Piping output of subprocess.Popen to files
  26. What’s the difference between Python’s subprocess.call and subprocess.run
  27. Launch IPython notebook with selected browser
  28. Sending ^C to Python subprocess objects on Windows
  29. Execute Commands Sequentially in Python?
  30. How to install and import Python modules at runtime

Leave a Comment