How to make file creation an atomic operation?

by

Write data to a temporary file and when data has been successfully written, rename the file to the correct destination file e.g

f = open(tmpFile, 'w')
f.write(text)
# make sure that all data is on disk
# see http://stackoverflow.com/questions/7433057/is-rename-without-fsync-safe
f.flush()
os.fsync(f.fileno()) 
f.close()

os.rename(tmpFile, myFile)

According to doc http://docs.python.org/library/os.html#os.rename

If successful, the renaming will be an atomic operation (this is a
POSIX requirement). On Windows, if dst
already exists, OSError will be raised
even if it is a file; there may be no
way to implement an atomic rename when
dst names an existing file

also

The operation may fail on some Unix flavors if src and dst are on different filesystems.

Note:

  • It may not be atomic operation if src and dest locations are not on same filesystem

  • os.fsync step may be skipped if performance/responsiveness is more important than the data integrity in cases like power failure, system crash etc

More Related Contents:

  1. python program to divide numbers in file in selected order [closed]
  2. How do I delete a file or folder in Python?
  3. UnicodeDecodeError: ‘charmap’ codec can’t decode byte X in position Y: character maps to
  4. How do I concatenate text files in Python?
  5. Get last n lines of a file, similar to tail
  6. How can I open multiple files using “with open” in Python?
  7. Read .mat files in Python
  8. Reading from a frequently updated file
  9. open() in Python does not create a file if it doesn’t exist
  10. Does reading an entire file leave the file handle open?
  11. How to read file N lines at a time?
  12. Python, Pandas : write content of DataFrame into text File
  13. How to delete specific strings from a file?
  14. Replace string within file contents [duplicate]
  15. Batch Renaming of Files in a Directory
  16. How to solve “OSError: telling position disabled by next() call”
  17. An integer is required? open()
  18. using Python logger class to generate multiple logs for different log levels
  19. Python Random Access File
  20. How to read tokens without reading whole line or file
  21. How can I read large text files line by line, without loading them into memory? [duplicate]
  22. Parsing apache log files
  23. Open files in ‘rt’ and ‘wt’ modes
  24. Trying to use open(filename, ‘w’ ) gives IOError: [Errno 2] No such file or directory if directory doesn’t exist
  25. Python file operations
  26. Speed up writing to files
  27. How to write to a file without overwriting current contents? [duplicate]
  28. python write string directly to tarfile
  29. Is a variable swap guaranteed to be atomic in python?
  30. Change file to read-only mode in Python

Leave a Comment