Remove duplicate entries in a Bash script [duplicate]

by

You can sort then uniq:

$ sort -u input.txt

Or use awk:

$ awk '!a[$0]++' input.txt

More Related Contents:

  1. Expansion of variables inside single quotes in a command in Bash
  2. Reading quoted/escaped arguments correctly from a string
  3. Can a shell script set environment variables of the calling shell? [duplicate]
  4. How to add a progress bar to a shell script?
  5. What is the purpose of “&&” in a shell command?
  6. How can I remove the extension of a filename in a shell script?
  7. sudo echo “something” >> /etc/privilegedFile doesn’t work [duplicate]
  8. While-loop subshell dilemma in Bash
  9. How do I escape the wildcard/asterisk character in bash?
  10. How do I grab an INI value within a shell script?
  11. How to create a cron job using Bash automatically without the interactive editor?
  12. How can I count all the lines of code in a directory recursively?
  13. How does Ctrl-C terminate a child process?
  14. Running multiple commands in one line in shell
  15. Division in script and floating-point
  16. Copy folder recursively, excluding some folders
  17. Variable interpolation in the shell
  18. How to remove filename prefix with a Posix shell
  19. How to pipe multiple commands into a single command in the shell? (sh, bash, …)
  20. What does ” 2>&1 ” mean?
  21. How do I read user input into a variable in Bash?
  22. Why do I get “/bin/sh: Argument list too long” when passing quoted arguments?
  23. Count number of lines in a git repository
  24. How do I check if a directory exists or not in a Bash shell script?
  25. shell script respond to keypress
  26. Temporarily change current working directory in bash to run a command [duplicate]
  27. How to break out of a loop in Bash?
  28. How to tell if a string is not defined in a Bash shell script
  29. BASH: Writing a Script to Recursively Travel a Directory of N Levels
  30. How to grep for case insensitive string in a file?

Leave a Comment