How can I loop over the output of a shell command?

by

Never for loop over the results of a shell command if you want to process it line by line unless you are changing the value of the internal field separator $IFS to \n. This is because the lines will get subject of word splitting which leads to the actual results you are seeing. Meaning if you for example have a file like this:

foo bar
hello world

The following for loop

for i in $(cat file); do
    echo "$i"
done

gives you:

foo
bar
hello
world

Even if you use IFS='\n' the lines might still get subject of Filename expansion

I recommend to use while + read instead because read reads line by line.

Furthermore I would use pgrep if you are searching for pids belonging to a certain binary. However, since python might appear as different binaries, like python2.7 or python3.4 I suggest to pass -f to pgrep which makes it search the whole command line rather than just searching for binaries called python. But this will also find processes which have been started like cat foo.py. You have been warned! At the end you can refine the regex passed to pgrep like you wish.

Example:

pgrep -f python | while read -r pid ; do
    echo "$pid"
done

or if you also want the process name:

pgrep -af python | while read -r line ; do
    echo "$line"
done

If you want the process name and the pid in separate variables:

pgrep -af python | while read -r pid cmd ; do
    echo "pid: $pid, cmd: $cmd"
done

You see, read offers a flexible and stable way to process the output of a command line-by-line.

Btw, if you prefer your ps .. | grep command line over pgrep use the following loop:

ps -ewo pid,etime,cmd | grep python | grep -v grep | grep -v sh \
  | while read -r pid etime cmd ; do
    echo "$pid $cmd $etime"
done

Note how I changed the order of etime and cmd. Thus to be able to read cmd, which can contain whitespace, into a single variable. This works because read will break down the line into variables, as many times as you specified variables. The remaining part of the line – possibly including whitespace – will get assigned to the last variable which has been specified in the command line.

More Related Contents:

  1. sh: 1: Syntax error: "(" unexpected error
  2. Iterate over a list of files with spaces
  3. How to declare 2D array in bash
  4. Defining a variable with or without export
  5. What does $@ mean in a shell script?
  6. The ‘eval’ command in Bash and its typical uses
  7. Better way to rename files based on multiple patterns
  8. How to join multiple lines of file names into one with custom delimiter?
  9. Syntax error in shell script with process substitution
  10. Raise error in a Bash script
  11. Linux: copy and create destination dir if it does not exist
  12. What is the difference between “#!/usr/bin/env bash” and “#!/usr/bin/bash”?
  13. How to handle more than 10 parameters in shell
  14. How to preserve line breaks when storing command output to a variable?
  15. Changing default shell in Linux [closed]
  16. How to use sed to remove the last n lines of a file
  17. Example of using named pipes in Linux shell (Bash)
  18. Passing variables in remote ssh command
  19. How do I run multiple background commands in bash in a single line?
  20. Execute command after every command in bash
  21. How do I change bash history completion to complete what’s already on the line?
  22. How to join multiple lines of filenames into one with custom delimiter
  23. Multithreading in Bash [duplicate]
  24. Running shell script in parallel
  25. Difference between shell and environment variables
  26. SUID not working with shell script
  27. What’s the point of eval/bash -c as opposed to just evaluating a variable?
  28. How to perform a for-each loop over all the files under a specified path?
  29. Write a bash shell script that consumes a constant amount of RAM for a user defined time [closed]
  30. How to create a dynamic variable and assign value to it?

Leave a Comment