Grepping a huge file (80GB) any way to speed it up?

by

Here are a few options:

1) Prefix your grep command with LC_ALL=C to use the C locale instead of UTF-8.

2) Use fgrep because you’re searching for a fixed string, not a regular expression.

3) Remove the -i option, if you don’t need it.

So your command becomes:

LC_ALL=C fgrep -A 5 -B 5 'db_pd.Clients' eightygigsfile.sql

It will also be faster if you copy your file to RAM disk.

More Related Contents:

  1. how does the bash command "grep -v ":0"" work
  2. How can I manipulate text in bash? [closed]
  3. How can I pipe stderr, and not stdout?
  4. How to merge every two lines into one from the command line?
  5. Fast way of finding lines in one file that are not in another?
  6. Capturing Groups From a Grep RegEx
  7. How to get the part of a file after the first line that matches a regular expression
  8. Is there an easy way to pass a “raw” string to grep?
  9. How to process each output line in a loop?
  10. Checking if output of a command contains a certain string in a shell script
  11. Colorized grep — viewing the entire file with highlighted matches
  12. How to check if a file contains a specific string using Bash
  13. Piping tail output though grep twice
  14. Check if all of multiple strings or regexes exist in a file
  15. Fastest possible grep
  16. Waiting for background processes to finish before exiting script
  17. How to give a pattern for new line in grep?
  18. Prevent grep returning an error when input doesn’t match
  19. Grep output with multiple Colors?
  20. Using grep to search for hex strings in a file
  21. How to strip out all of the links of an HTML file in Bash or grep or batch and store them in a text file
  22. How to delete rows from a csv file based on a list values from another file?
  23. How to grep for case insensitive string in a file?
  24. Convert absolute path into relative path given a current directory using Bash
  25. Convert date time string to epoch in Bash
  26. How do I put an already-running process under nohup?
  27. How to obtain the absolute path of a file via Shell (BASH/ZSH/SH)?
  28. Loop only iterates once with `ssh` in the body [duplicate]
  29. Send command to a background process
  30. What does — do when running an npm command?

Leave a Comment