When should I use mmap for file access?

by

mmap is great if you have multiple processes accessing data in a read only fashion from the same file, which is common in the kind of server systems I write. mmap allows all those processes to share the same physical memory pages, saving a lot of memory.

mmap also allows the operating system to optimize paging operations. For example, consider two programs; program A which reads in a 1MB file into a buffer creating with malloc, and program B which mmaps the 1MB file into memory. If the operating system has to swap part of A‘s memory out, it must write the contents of the buffer to swap before it can reuse the memory. In B‘s case any unmodified mmap‘d pages can be reused immediately because the OS knows how to restore them from the existing file they were mmap‘d from. (The OS can detect which pages are unmodified by initially marking writable mmap‘d pages as read only and catching seg faults, similar to Copy on Write strategy).

mmap is also useful for inter process communication. You can mmap a file as read / write in the processes that need to communicate and then use synchronization primitives in the mmap'd region (this is what the MAP_HASSEMAPHORE flag is for).

One place mmap can be awkward is if you need to work with very large files on a 32 bit machine. This is because mmap has to find a contiguous block of addresses in your process’s address space that is large enough to fit the entire range of the file being mapped. This can become a problem if your address space becomes fragmented, where you might have 2 GB of address space free, but no individual range of it can fit a 1 GB file mapping. In this case you may have to map the file in smaller chunks than you would like to make it fit.

Another potential awkwardness with mmap as a replacement for read / write is that you have to start your mapping on offsets of the page size. If you just want to get some data at offset X you will need to fixup that offset so it’s compatible with mmap.

And finally, read / write are the only way you can work with some types of files. mmap can’t be used on things like pipes and ttys.

More Related Contents:

  1. Atomicity of `write(2)` to a local filesystem
  2. Linux shared memory: shmget() vs mmap()?
  3. Read/write files within a Linux kernel module
  4. What is the difference between sigaction and signal?
  5. Why does unaligned access to mmap’ed memory sometimes segfault on AMD64?
  6. Why is fseek or fflush always required between reading and writing in the update modes?
  7. read comma-separated input with `scanf()`
  8. C write in the middle of a binary file without overwriting any existing content
  9. GCC with -std=c99 complains about not knowing struct timespec
  10. Does printf always flush the buffer on encountering a newline?
  11. What does “#define _GNU_SOURCE” imply?
  12. What happens if I don’t call fclose() in a C program?
  13. Unexpected exec permission from mmap when assembly files included in the project
  14. File Operations in Android NDK
  15. Why does open() create my file with the wrong permissions?
  16. Is there a way to flush a POSIX socket?
  17. How to redirect the output back to the screen after freopen(“out.txt”, “a”, stdout)
  18. Recursive mkdir() system call on Unix
  19. Why is the fgets function deprecated?
  20. Mmap() an entire large file
  21. Linux/POSIX equivalent for Win32’s CreateEvent, SetEvent, WaitForSingleObject
  22. How do I find the current machine’s full hostname in C (hostname and domain information)?
  23. Sharing memory between processes through the use of mmap()
  24. Is it possible (and safe) to make an accepting socket non-blocking?
  25. Posix regular expression non-greedy
  26. Difference between C standard library and C POSIX library
  27. Reading a file character by character in C
  28. ftell at a position past 2GB
  29. Tried and true simple file copying code in C?
  30. How do I insert and delete some characters in the middle of a file?

Leave a Comment