C++ cout and cin buffers, and buffers in general

The basic idea of buffering is to combine operations into bigger chunks: instead of reading small number of bytes, read an entire page and make this available as requested; instead of writing small number of bytes, buffer them up and write an entire page or when writing is explicitly requested. Essentially, this is an important performance optimization. It pretty clear cut for I/O operations but generally applies for other uses as well: processing multiple units at once generally ends up to be faster than processing individual units.

With respect to I/O flushing refers to writing the currently buffered bytes to its destination – whatever this means in practice. For C++ IOStreams flushing a stream amounts to calling the member function std::ostream::flush() which in turn calls std::streambuf::pubsync() on the associated stream buffer (this ignores the detail that the stream are actually class templates, e.g. std::basic_ostream<cT, traits>; for the purpose of this discussion it doesn’t matter that they are class templates): the base class std::streambuf is C++’s abstraction on how a certain stream is to be processed. It conceptually consists of an input and an output buffer plus virtual function responsible for reading or writing the buffers, respectively. The function std::streambuf::pubsync() calls the virtual function std::streambuf::sync() which should be overridden for every stream buffer which potentially buffers characters. That is, what flushing actually means depends on how this virtual function is implemented.

Whether an override of sync() actually does something and what it does clearly depends on what the stream buffer represents. For example, for a std::filebuf which is responsible to reading from or writing to a file, sync() writes the current content of the buffer and removes the flushed characters from the buffer. Given that a file may not really represent a physical file but e.g. a named pipe to communicate with a different process, this is reasonable behavior. On the other hand, flushing a std::stringbuf which is the stream buffer used to implement writing to a std::string used e.g. by std::ostringstream actually doesn’t do anything: the string is entirely within the program and the std::string representing its value is constructed when the std::stringbuf::str() member function is called. For user defined stream buffers there are many different behaviors. A common class of stream buffers is filtering the output somehow before passing it on to another stream buffer (e.g. a logging buffer may add a time stamp after each newline). These typically just call the std::streambuf::pubsync() function of the next stream buffers.

Thus, descriptions of the actual behavior are typically kept fairly vague because it isn’t really clear what exactly happens. Conceptually, flushing a stream or calling pubsync() on a stream buffer should update the external destination of the character to match the current internal state. Generally, this amounts to forwarding the currently buffered characters and removing them from the internal buffer. At this point it is worth noting that the buffer typically also gets written when it is just full. When it gets full depends, again, on the particular stream buffer. A good implementation of std::filebuf will essentially fill a buffer of bytes matching the size of the underlying page (or a multiple thereof) and then write complete pages, minimizing the number of I/O operations needed (this is actually relatively tricky to do because buffer sizes are different between different file systems and depending on the encoding used when writing the number of produced bytes can’t be easily estimated).

The standard C++ library typically doesn’t require explicit flushes:

• The stream std::cerr is set up to automatically flush any output produced after each output operations being called. This is the result of the formatting flag std::ios_base::unitbuf being set by default. To turn this off you can use std::cerr << std::nounitbuf or your can just use std::clog which writes to the same destination but doesn’t to do this flushing.
• When reading from an std::istream the “tied” std::ostream, if any, is flushed. By default std::cout is tied to std::cin. If you want to set up a tied std::ostream yourself you can use e.g. in.tie(&out) or, if you want to remove a tied std::ostream you can use e.g. std::cin.tie(0).
• When an std::ostream is destroyed (or when it would normally destroyed in case the std::ostream is one of the standard streams), the std::ostream is flushed.
• When a stream’s buffer would overflow the virtual function std::streambuf::overflow() is called which typically writes the buffer of the current buffer (plus the passed character, if any) to its destination. This often is done by just calling sync() to clear out the current buffer but what is done exactly, again, depends on the concrete stream buffer.

You can also explicitly request flushing an std::ostream:

• You can call the member function std::ostream::flush(), e.g. std::cout.flush().
• You can call the member function std::streambuf::pubsync(), e.g. std::cout.rdbuf()->pubsync() (assuming there is a stream buffer set up; calling std::ostream::flush() will do nothing if there is no stream buffer).
• You can use the manipulator std::flush, e.g. std::cout << std::flush.
• You can use the manipulator std::endl, e.g. std::cout << std::endl to first write a newline character followed by flushing the stream. Note that you should use std::endl only when you really mean to flush the output. Do not use std::endl when you actually just want to create an end of line! Just write a newline character for the latter!

In any case, if you just write a couple of characters which don’t cause the stream buffer’s buffer to overflow, nothing will happen with them until they are either implicitly or explicitly flushed. Generally, this isn’t a problem because the normal case where buffered output needs to become available, i.e. when reading from std::cin, is handled by std::cout being tie()d to std::cin. When there are other I/O mechanisms being used it may be necessary to explicitly tie() related streams. The buffer is sometimes a problem if the program “crashes” or asserts during debugging because some outputs to a stream may have been made but weren’t, yet, sent. The remedy for this is to use std::unitbuf.