Safe to have multiple processes writing to the same file at the same time? [CentOs 6, ext4]

What you’re doing seems perfectly OK, provided you’re using the POSIX “raw” IO syscalls such as read(), write(), lseek() and so forth.

If you use C stdio (fread(), fwrite() and friends) or some other language runtime library which has its own userspace buffering, then the answer by “Tilo” is relevant, in that due to the buffering, which is to some extent outside your control, the different processes might overwrite each other’s data.

Wrt OS locking, while POSIX states that writes or reads less than of size PIPE_BUF are atomic for some special files (pipes and FIFO’s), there is no such guarantee for regular files. In practice, I think it’s likely that IO’s within a page are atomic, but there is no such guarantee. The OS only does locking internally to the extent that is necessary to protect its own internal data structures. One can use file locks, or some other interprocess communication mechanism, to serialize access to files. But, all this is relevant only of you have several processes doing IO to the same region of a file. In your case, as your processes are doing IO to disjoint sections of the file, none of this matters, and you should be fine.

Leave a Comment