What exactly is file.flush() doing?

by

There’s typically two levels of buffering involved:

  1. Internal buffers
  2. Operating system buffers

The internal buffers are buffers created by the runtime/library/language that you’re programming against and is meant to speed things up by avoiding system calls for every write. Instead, when you write to a file object, you write into its buffer, and whenever the buffer fills up, the data is written to the actual file using system calls.

However, due to the operating system buffers, this might not mean that the data is written to disk. It may just mean that the data is copied from the buffers maintained by your runtime into the buffers maintained by the operating system.

If you write something, and it ends up in the buffer (only), and the power is cut to your machine, that data is not on disk when the machine turns off.

So, in order to help with that you have the flush and fsync methods, on their respective objects.

The first, flush, will simply write out any data that lingers in a program buffer to the actual file. Typically this means that the data will be copied from the program buffer to the operating system buffer.

Specifically what this means is that if another process has that same file open for reading, it will be able to access the data you just flushed to the file. However, it does not necessarily mean it has been “permanently” stored on disk.

To do that, you need to call the os.fsync method which ensures all operating system buffers are synchronized with the storage devices they’re for, in other words, that method will copy data from the operating system buffers to the disk.

Typically you don’t need to bother with either method, but if you’re in a scenario where paranoia about what actually ends up on disk is a good thing, you should make both calls as instructed.

Addendum in 2018.

Note that disks with cache mechanisms is now much more common than back in 2013, so now there are even more levels of caching and buffers involved. I assume these buffers will be handled by the sync/flush calls as well, but I don’t really know.

More Related Contents:

  1. Why can’t I call read() twice on an open file?
  2. A non-blocking read on a subprocess.PIPE in Python
  3. Confused by python file mode “w+”
  4. How to redirect ‘print’ output to a file?
  5. Unicode (UTF-8) reading and writing to files in Python
  6. Editing specific line in text file in Python
  7. How to write to a CSV line by line?
  8. Reading file using relative path in python project
  9. Python multiprocessing safely writing to a file
  10. Load data from txt with pandas
  11. Beginner Python: Reading and writing to the same file
  12. Read a small random sample from a big CSV file into a Python data frame
  13. How to read bits from a file?
  14. Using “readlines()” twice in a row [duplicate]
  15. How to create in-memory file object
  16. Python – IOError: [Errno 13] Permission denied:
  17. What’s the difference between io.open() and os.open() on Python?
  18. Python code performance decreases with threading
  19. parsing a tab-separated file in Python
  20. reading v 7.3 mat file in python
  21. IPython: redirecting output of a Python script to a file (like bash >)
  22. Reading a file with a specified delimiter for newline
  23. Tkinter GUI, I/O & Threading: When to use queues, when events?
  24. Why doesn’t print output show up immediately in the terminal when there is no newline at the end?
  25. Fastest save and load options for a numpy array
  26. When should I ever use file.read() or file.readlines()?
  27. Why is `with open()` better for opening files in Python?
  28. Binary buffer in Python
  29. Reading a File Line by Line in Python [duplicate]
  30. In Python, is read() , or readlines() faster?

Leave a Comment