How can I split a large text file into smaller files with an equal number of lines?

by

Have a look at the split command:

$ split --help
Usage: split [OPTION] [INPUT [PREFIX]]
Output fixed-size pieces of INPUT to PREFIXaa, PREFIXab, ...; default
size is 1000 lines, and default PREFIX is `x'.  With no INPUT, or when INPUT
is -, read standard input.

Mandatory arguments to long options are mandatory for short options too.
  -a, --suffix-length=N   use suffixes of length N (default 2)
  -b, --bytes=SIZE        put SIZE bytes per output file
  -C, --line-bytes=SIZE   put at most SIZE bytes of lines per output file
  -d, --numeric-suffixes  use numeric suffixes instead of alphabetic
  -l, --lines=NUMBER      put NUMBER lines per output file
      --verbose           print a diagnostic to standard error just
                            before each output file is opened
      --help     display this help and exit
      --version  output version information and exit

You could do something like this:

split -l 200000 filename

which will create files each with 200000 lines named xaa xab xac …

Another option, split by size of output file (still splits on line breaks):

 split -C 20m --numeric-suffixes input_filename output_prefix

creates files like output_prefix01 output_prefix02 output_prefix03 ... each of maximum size 20 megabytes.

More Related Contents:

  1. Print a file’s last modified date in Bash
  2. How can I check if a directory exists in a Bash shell script?
  3. What is the cleanest way to ssh and run multiple commands in Bash?
  4. Bash tool to get nth line from a file
  5. How to remove files starting with double hyphen?
  6. How to get the primary IP address of the local machine on Linux and OS X? [closed]
  7. How to merge two files line by line in Bash
  8. How to suppress Terminated message after killing in bash?
  9. How to make the ‘cut’ command treat same sequental delimiters as one?
  10. Force flushing of output to a file while bash script is still running
  11. How do I get bc(1) to print the leading zero?
  12. Delete files older than 10 days using shell script in Unix [duplicate]
  13. What do double-asterisk (**) wildcards mean?
  14. Copy folder recursively, excluding some folders
  15. Variable interpolation in the shell
  16. Creating a string variable name from the value of another string
  17. How do I pause my shell script for a second before continuing?
  18. Is there a way to make bash job control quiet?
  19. What does ” 2>&1 ” mean?
  20. What does the double-asterisk (**) wildcard mean?
  21. Bash regex =~ operator
  22. How do I check if a directory exists or not in a Bash shell script?
  23. Bash script to receive and repass quoted parameters
  24. Convert Unix timestamp to a date string
  25. Using Bash to display a progress indicator (spinner) [duplicate]
  26. Why doesn’t **sort** sort the same on every machine?
  27. Extract version number from a string
  28. escaping newlines in sed replacement string
  29. How to wait on all child (and grandchild etc) process spawned by a script
  30. Subtract two variables in Bash

Leave a Comment