How can I get unique values from an array in Bash?

by

A bit hacky, but this should do it:

echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '

To save the sorted unique results back into an array, do Array assignment:

sorted_unique_ids=($(echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' '))

If your shell supports herestrings (bash should), you can spare an echo process by altering it to:

tr ' ' '\n' <<< "${ids[@]}" | sort -u | tr '\n' ' '

A note as of Aug 28 2021:

According to ShellCheck wiki 2207 a read -a pipe should be used to avoid splitting.
Thus, in bash the command would be:

IFS=" " read -r -a ids <<< "$(echo "${ids[@]}" | tr ' ' '\n' | sort -u | tr '\n' ' ')"

or

IFS=" " read -r -a ids <<< "$(tr ' ' '\n' <<< "${ids[@]}" | sort -u | tr '\n' ' ')"

Input:

ids=(aa ab aa ac aa ad)

Output:

aa ab ac ad

Explanation:

  • "${ids[@]}" – Syntax for working with shell arrays, whether used as part of echo or a herestring. The @ part means “all elements in the array”
  • tr ' ' '\n' – Convert all spaces to newlines. Because your array is seen by shell as elements on a single line, separated by spaces; and because sort expects input to be on separate lines.
  • sort -u – sort and retain only unique elements
  • tr '\n' ' ' – convert the newlines we added in earlier back to spaces.
  • $(...) – Command Substitution
  • Aside: tr ' ' '\n' <<< "${ids[@]}" is a more efficient way of doing: echo "${ids[@]}" | tr ' ' '\n'

More Related Contents:

  1. Why should eval be avoided in Bash, and what should I use instead?
  2. How to redirect output to a file and stdout
  3. How to kill a child process after a given timeout in Bash?
  4. ./configure : /bin/sh^M : bad interpreter [duplicate]
  5. The ‘eval’ command in Bash and its typical uses
  6. How to join multiple lines of file names into one with custom delimiter?
  7. Linux: copy and create destination dir if it does not exist
  8. How to redirect the output of the time command to a file in Linux?
  9. ‘&&’ vs. ‘&’ with the ‘test’ command in Bash
  10. How to redirect stderr and stdout to different files in the same line in script?
  11. How to use ‘cp’ command to exclude a specific directory?
  12. Linux bash: Multiple variable assignment
  13. How to sort a file in-place?
  14. What is the meaning of a double dollar sign in bash/Makefile?
  15. Is it OK to use the same input file as output of a piped command?
  16. Run cron job only if it isn’t already running
  17. Redirecting command output in docker
  18. When to wrap quotes around a shell variable?
  19. How to get file creation date/time in Bash/Debian?
  20. Disable and re-enable address space layout randomization only for myself
  21. How to join multiple lines of filenames into one with custom delimiter
  22. Check if a variable exists in a list in Bash
  23. Doesn’t sh support process substitution
  24. How to automatically add user account AND password with a Bash script?
  25. Meaning of tilde in Linux bash (not home directory)
  26. How do I delete all lines in a file starting from after a matching line?
  27. How to convert hex to ASCII characters in the Linux shell?
  28. How to check if a program is run in Bash on Ubuntu on Windows and not just plain Ubuntu?
  29. What are the rules for valid identifiers (e.g. functions, vars, etc) in Bash?
  30. Bash `wait` command, waiting for more than 1 PID to finish execution

Leave a Comment