Fastest Way to Delete a Line from Large File in Python

by

You can have two file objects for the same file at the same time (one for reading, one for writing):

def removeLine(filename, lineno):
    fro = open(filename, "rb")

    current_line = 0
    while current_line < lineno:
        fro.readline()
        current_line += 1

    seekpoint = fro.tell()
    frw = open(filename, "r+b")
    frw.seek(seekpoint, 0)

    # read the line we want to discard
    fro.readline()

    # now move the rest of the lines in the file 
    # one line back 
    chars = fro.readline()
    while chars:
        frw.writelines(chars)
        chars = fro.readline()

    fro.close()
    frw.truncate()
    frw.close()

More Related Contents:

  1. Fastest way to list all primes below N
  2. A weighted version of random.choice
  3. How to improve performance of this code?
  4. Handling very large numbers in Python
  5. What is the fastest way to flatten arbitrarily nested lists in Python? [duplicate]
  6. Flatten an irregular list of lists
  7. Why is my NLTK function slow when processing the DataFrame?
  8. How to iterate over a list in chunks
  9. When is not a good time to use python generators?
  10. How to split/partition a dataset into training and test datasets for, e.g., cross validation?
  11. Python equivalent of MATLAB’s “ismember” function
  12. Python vectorizing nested for loops
  13. How to efficiently use MySQLDB SScursor?
  14. What does Python optimization (-O or PYTHONOPTIMIZE) do?
  15. Efficient way to take the minimum/maximum n values and indices from a matrix using NumPy
  16. How do I put a constraint on SciPy curve fit?
  17. Processing time gets longer and longer after each iteration (TensorFlow)
  18. Most efficient way to calculate radial profile
  19. scipy is not optimizing and returns “Desired error not necessarily achieved due to precision loss”
  20. sorted() using generator expressions rather than lists
  21. Flatten an irregular (arbitrarily nested) list of lists
  22. Best way to loop over a python string backwards
  23. Concatenate range arrays given start, stop numbers in a vectorized way – NumPy
  24. Multiplying a huge number times random() (Python)
  25. Scikit Learn GridSearchCV without cross validation (unsupervised learning)
  26. Techniques for working with large Numpy arrays? [duplicate]
  27. First items in inner list efficiently as possible [duplicate]
  28. Python equivalence to inline functions or macros
  29. Elegant way to remove items from sequence in Python? [duplicate]
  30. os.walk very slow, any way to optimise?

Leave a Comment