Working with huge files in VIM

by

I had a 12GB file to edit today. The vim LargeFile plugin did not work for me. It still used up all my memory and then printed an error message :-(. I could not use hexedit for either, as it cannot insert anything, just overwrite. Here is an alternative approach:

You split the file, edit the parts and then recombine it. You still need twice the disk space though.

  • Grep for something surrounding the line you would like to edit:

    grep -n 'something' HUGEFILE | head -n 1

  • Extract that range of the file. Say the lines you want to edit are at line 4 and 5. Then do:

    sed -n -e '4,5p' -e '5q' HUGEFILE > SMALLPART
    • The -n option is required to suppress the default behaviour of sed to print everything
    • 4,5p prints lines 4 and 5
    • 5q aborts sed after processing line 5

  • Edit SMALLPART using your favourite editor.

  • Combine the file:

    (head -n 3 HUGEFILE; cat SMALLPART; sed -e '1,5d' HUGEFILE) > HUGEFILE.new
    • i.e: pick all the lines before the edited lines from the HUGEFILE (which in this case is the top 3 lines), combine it with the edited lines (in this case lines 4 and 5) and use this combined set of lines to replace the equivalent (in this case the top 5 lines) in the HUGEFILE and write it all to a new file.

    HUGEFILE.new will now be your edited file, you can delete the original HUGEFILE.

More Related Contents:

  1. How do I exit the Vim editor?
  2. What is your most productive shortcut with Vim?
  3. Using Vim’s tabs like buffers
  4. How to paste yanked text into the Vim command line
  5. Vim clear last search highlighting
  6. Why should I use an IDE? [closed]
  7. Turning off auto indent when pasting text into vim
  8. Where is my .vimrc file?
  9. Search for selection in Vim
  10. How to paste over without overwriting register
  11. A more useful statusline in vim? [closed]
  12. How can I automatically add some skeleton code when creating a new file with vim
  13. How to add line numbers to range of lines in Vim?
  14. Is there a “vim runtime log”?
  15. Vim auto-generate ctags
  16. How to prevent the cursor from moving back one character on leaving Insert mode in Vim?
  17. Using vim’s f command over multiple lines
  18. How to effectively work with multiple files in Vim
  19. Why is \r a newline for Vim?
  20. reformat in vim for a nice column layout
  21. Best way to insert timestamp in Vim?
  22. Automatic closing brackets for Vim [closed]
  23. Remove all arbitary spaces before a line in Vim
  24. Can terminals detect or ?
  25. How do I fix the indentation of an entire file in Vi?
  26. Replace tabs with spaces in vim
  27. Multiple commands on same line
  28. How can you automatically remove trailing whitespace in vim
  29. Setting Vim whitespace preferences by filetype
  30. Paste in insert mode?

Leave a Comment