Run several jobs parallelly and Efficiently

by

As Mark Setchell says: GNU Parallel.

find scripts/ -type f | parallel

If you insists on keeping 8 CPUs free:

find scripts/ -type f | parallel -j-8

But usually it is more efficient simply to use nice as that will give you all 48 cores when no one else needs them:

find scripts/ -type f | nice -n 15 parallel

To learn more:

More Related Contents:

  1. Running shell script in parallel
  2. How do I send a file as an email attachment using Linux command line?
  3. What does the number in parentheses shown after Unix command names in manpages mean?
  4. How to install latest version of git on CentOS 8.x/7.x/6.x
  5. Docker can’t connect to docker daemon
  6. Adding a new entry to the PATH variable in ZSH
  7. How do I get the total CPU usage of an application from /proc/pid/stat?
  8. How can I delete a newline if it is the last character in a file?
  9. Makefiles with source files in different directories
  10. Use of floating point in the Linux kernel
  11. Error when using Git credential helper with gnome-keyring as Sudo
  12. How do I change permissions for a folder and its subfolders/files?
  13. How can I generate a list of files with their absolute path in Linux?
  14. How to print a character in Linux x86 NASM?
  15. Pipe output to bash function
  16. List files recursively in Linux CLI with path relative to the current directory
  17. rdtsc accuracy across CPU cores
  18. How to escape a single quote in single quote string in Bash?
  19. Prevent Gnome Terminal From Exiting After Execution [duplicate]
  20. How do I SET the GOPATH environment variable on Ubuntu? What file must I edit?
  21. Monitor Directory for Changes
  22. Trying to understand gcc’s complicated stack-alignment at the top of main that copies the return address
  23. How to kill zombie process
  24. How can I trim white space from a variable in awk?
  25. ARM inline asm: exit system call with value read from memory
  26. Alias with variable in bash [duplicate]
  27. How find out which process is using a file in Linux?
  28. Creating temporary files in bash
  29. Maximum number of concurrent connections on a single port (socket) of Server
  30. Changing environment variable of a running process

Leave a Comment