This is the tenth video in Part 3 of the Performance-Aware Programming series. Please see the Table of Contents to quickly navigate through the rest of the course as it is updated weekly. The listings referenced in the video (listings 129 and 130) are available on the github.
I thought we were done with page mapping topics last week, but, several people have since asked about memory-mapped files — an extension of the OS’s page mapping that does automatic file I/O — so we’re going to take a brief look at those today. But after this I am putting my foot down and saying, “no more page mapping!” We’re moving on to CPU internals!
Thankfully, there isn’t much more to learn in order to fully understand memory-mapped files. Since you already know how four-level paging works, you already know most of what you need.
Remember how page faults worked? The OS would wake up on a page fault interrupt whenever our thread touched an unmapped page, and it would map it to a clear page of physical memory before allowing our thread to resume execution. Memory mapped files work the same way, except instead of using a clear page of physical memory, the OS first reads part of a file into the physical memory. This behavior allows the OS to provide “read-on-demand” memory that mirrors the contents of a file you request. Using the page dirty bits, it can also write back modifications to the file on a page-by-page basis.
This is effectively a “lazy file I/O” scheme. Normally, if you wanted to read a file, you would call two APIs: VirtualAlloc to map some memory space to hold the file, and ReadFile to copy the contents of the file into the newly-allocated memory. With memory-mapped files, you do both at the same time: the MapViewOfFile API reserves a virtual address range for storing the file, but the OS remembers that it is going to be used specifically for mirroring the contents of that file. When you try to read from any piece of that memory, the resulting page fault will cause the OS to read the corresponding page of the file automatically.
That’s really all there is to it. The code to memory-map a file on Windows is trivial: