file.tell() inconsistency

by

Using open files as an iterator uses a read-ahead buffer to increase efficiency. As a result, the file pointer advances in large steps across the file as you loop over the lines.

From the File Objects documentation:

In order to make a for loop the most efficient way of looping over the lines of a file (a very common operation), the next() method uses a hidden read-ahead buffer. As a consequence of using a read-ahead buffer, combining next() with other file methods (like readline()) does not work right. However, using seek() to reposition the file to an absolute position will flush the read-ahead buffer.

If you need to rely on .tell(), don’t use the file object as an iterator. You can turn .readline() into an iterator instead (at the price of some performance loss):

for line in iter(f.readline, ''):
    print f.tell()

This uses the iter() function sentinel argument to turn any callable into an iterator.

More Related Contents:

  1. Writing a Python list of lists to a csv file
  2. Reading a huge .csv file
  3. Fastest way to process a large file?
  4. Python 2.7: Print to File
  5. Is there a need to close files that have no reference to them?
  6. Mixing files and loops
  7. Sending a file over TCP sockets in Python
  8. How to read the last line of a file in Python? [duplicate]
  9. Python not able to open file with non-english characters in path
  10. Python 2.7 IndentationError [closed]
  11. How to rename a file in python
  12. How to detect key presses?
  13. How to read specific lines from a file (by line number)?
  14. Using both Python 2.x and Python 3.x in IPython Notebook
  15. List vs generator comprehension speed with join function [duplicate]
  16. Launch a completely independent process
  17. Why does str(float) return more digits in Python 3 than Python 2?
  18. Add pygame module in PyCharm IDE
  19. How to pass an operator to a python function?
  20. Running powershell script within python script, how to make python print the powershell output while it is running
  21. Print LIST of unicode chars without escape characters
  22. Does filehandle get closed automatically in Python after it goes out of scope?
  23. Python3 correct way to import relative or absolute?
  24. Use Multiple Character Delimiter in Python Pandas read_csv
  25. BeautifulSoup returns empty list when searching by compound class names
  26. Pip freeze vs. pip list
  27. how to merge two data frames based on particular column in pandas python?
  28. Catching exception in context manager __enter__()
  29. confused about random_state in decision tree of scikit learn
  30. Is possible to mapping view with class using mapper in SqlAlchemy?

Leave a Comment