Understanding Ruby and OS I/O buffering

by

The Ruby IO documentation is not 100% clear on how this buffering works, but this is what you can extract from the documentation:

  • Ruby IO has its own internal buffer
  • In addition to that the underlying operating system may or may not further buffer data.

The relevant methods to look at:

  • IO.flush: Flushes IO. I also looked at the Ruby source and a call to IO.flush also calls the underlying OS fflush(). This should be enough to get the file cached, but does not guarantee physical data to disk.
  • IO.sync=: If set to true, no Ruby internal buffering is done. Everything is immidiately sent to the OS, and fflush() is called for each write.
  • IO.sync: Returns the current sync setting (true or false).
  • IO.fsync: Flushes both the Ruby buffers + calls fsync() on the OS (if it supports it). This will guarantee a full flush all the way to the physical disk file.
  • IO.close: Closes the Ruby IO and writes pending data to the OS. Note that this does not imply fsync(). The POSIX documentation on close() says that it does NOT guarantee data is physically written to the file. So you need to use an explicit fsync() call for that.

Conclusion: flush and/or close should be enough to get the file cached so that it can be read fully by another process or operation. To get the file all the way to the physical media with certainty, you need to call IO.fsync.

Other related methods:

  • IO.syswrite: Bypass Ruby internal buffers and do a straight OS write. If you use this then do not mix it with IO.read/write.
  • IO.sysread: Same as above, but for reading.

More Related Contents:

  1. Why is “slurping” a file not a good practice?
  2. How can I find which operating system my Ruby program is running on?
  3. How to create a file in Ruby
  4. Set socket timeout in Ruby via SO_RCVTIMEO socket option
  5. Read, edit, and write a text file line-wise using Ruby
  6. How to convert Time format in ruby
  7. Regex for reverse match a string
  8. Why do Ruby setters need “self.” qualification within the class?
  9. Ruby, Difference between exec, system and %x() or Backticks
  10. Why does Ruby 1.9.2 remove “.” from LOAD_PATH, and what’s the alternative?
  11. rvm installation not working: “RVM is not a function”
  12. How to understand symbols in Ruby
  13. Ruby array access 2 consecutive(chained) elements at a time
  14. What does &. (ampersand dot) mean in Ruby?
  15. Should Gemfile.lock be included in .gitignore?
  16. Get names of all files from a folder with Ruby
  17. Unresolved specs during Gem::Specification.reset:
  18. Extract a substring from a string in Ruby using a regular expression
  19. How to convert a string to a constant in Ruby?
  20. What does the “||=” operand stand for in ruby [duplicate]
  21. Method to parse HTML document in Ruby?
  22. Really Cheap Command-Line Option Parsing in Ruby
  23. Running another ruby script from a ruby script
  24. ruby: class instance variables vs instance variables
  25. Ruby: Proc#call vs yield
  26. What is the use case for Ruby’s %q / %Q quoting methods?
  27. How to install ruby-debug when needing necessary libraries and/or headers
  28. How to cleanly initialize attributes in Ruby with new?
  29. Mac Rmagick won’t install with Xcode 4.2
  30. unzip (zip, tar, tag.gz) files with ruby

Leave a Comment