You’re being too conservative: A memory-mapped file can be larger than the address space. The view of the memory-mapped file is limited by OS memory constraints, but that’s only the part of the file you’re looking at at one time. (And I guess technically you could map multiple views of discontinuous parts of the file … Read more
mmap works differently. It is anticipatory and adapts to the program’s access pattern. Also, specific policies can be set via madvise to further fine tune usage. For a much more thorough discussion of how mmap works in a demand paging environment, see my answer here: Which segments are affected by a copy-on-write? as it also … Read more
Linux has an execution domain called READ_IMPLIES_EXEC, which causes all pages allocated with PROT_READ to also be given PROT_EXEC. Older Linux kernels used to use this for executables that used the equivalent of gcc -z execstack. This program will show you whether that’s enabled for itself: #include <stdio.h> #include <sys/personality.h> int main(void) { printf(“Read-implies-exec is … Read more
mmap is great if you have multiple processes accessing data in a read only fashion from the same file, which is common in the kind of server systems I write. mmap allows all those processes to share the same physical memory pages, saving a lot of memory. mmap also allows the operating system to optimize … Read more
The following analysis applies only to glibc (based on the ptmalloc2 algorithm). There are certain options that seem helpful to return the freed memory back to the system: mallopt() (defined in malloc.h) does provide an option to set the trim threshold value using one of the parameter option M_TRIM_THRESHOLD, this indicates the minimum amount of … Read more
I was trying to find the final word on mmap / read performance on Linux and I came across a nice post (link) on the Linux kernel mailing list. It’s from 2000, so there have been many improvements to IO and virtual memory in the kernel since then, but it nicely explains the reason why … Read more