Is file append atomic in UNIX?

by

A write that’s under the size of ‘PIPE_BUF’ is supposed to be atomic. That should be at least 512 bytes, though it could easily be larger (linux seems to have it set to 4096).

This assume that you’re talking all fully POSIX-compliant components. For instance, this isn’t true on NFS.

But assuming you write to a log file you opened in ‘O_APPEND’ mode and keep your lines (including newline) under ‘PIPE_BUF’ bytes long, you should be able to have multiple writers to a log file without any corruption issues. Any interrupts will arrive before or after the write, not in the middle. If you want file integrity to survive a reboot you’ll also need to call fsync(2) after every write, but that’s terrible for performance.

Clarification: read the comments and Oz Solomon’s answer. I’m not sure that O_APPEND is supposed to have that PIPE_BUF size atomicity. It’s entirely possible that it’s just how Linux implemented write(), or it may be due to the underlying filesystem’s block sizes.

More Related Contents:

  1. Differences between System V and Posix semaphores
  2. How can I check if a directory exists in a Bash shell script?
  3. Remove carriage return in Unix
  4. How to delete duplicate lines in a file without sorting it in Unix
  5. Force line-buffering of stdout in a pipeline
  6. Why is 1/1/1970 the “epoch time”?
  7. How to make file creation an atomic operation?
  8. Atomicity of `write(2)` to a local filesystem
  9. Grep for literal strings
  10. Use grep –exclude/–include syntax to not grep through certain files
  11. Delete specific line number(s) from a text file using sed?
  12. How does grep run so fast?
  13. How to do a mass rename?
  14. Date arithmetic in Unix shell scripts
  15. How does SIGINT relate to the other termination signals such as SIGTERM, SIGQUIT and SIGKILL?
  16. Why does open() create my file with the wrong permissions?
  17. Get last field using awk substr
  18. Recursive mkdir() system call on Unix
  19. unix command line execute with . (dot) vs. without
  20. How do you recursively unzip archives in a directory and its subdirectories from the Unix command-line?
  21. Unix sort of version numbers
  22. Remove non-ASCII characters in a file [duplicate]
  23. A cron job that will never execute
  24. Why does a read-only open of a named pipe block?
  25. Creating string of repeated characters in shell script [duplicate]
  26. Configure cron job to run every 15 minutes on Jenkins
  27. Grepping for exact words with UNIX
  28. Why does RSA encrypted text give me different results for the same text
  29. How to remove blank lines from a Unix file
  30. How to determine a terminal’s background color?

Leave a Comment