How big can a memory-mapped file be?

Solution 1:

You're being too conservative: A memory-mapped file can be larger than the address space. The view of the memory-mapped file is limited by OS memory constraints, but that's only the part of the file you're looking at at one time. (And I guess technically you could map multiple views of discontinuous parts of the file at once, so aside from overhead and page length constraints, it's only the total # of bytes you're looking at that poses a limit. You could look at bytes [0 to 1024] and bytes [2⁴⁰ to 2⁴⁰ + 1024] with two separate views.)

In MS Windows, look at the MapViewOfFile function. It effectively takes a 64-bit file offset and a 32-bit length.

Solution 2:

This has been my experience when using memory-mapped files under Win32:

If your map the entire file into one segment, it normally taps out at around 750 MB, because it can't find a bigger contiguous block of memory. If you split it up into smaller segments, say 100MB each, you can get around 1500MB-1800MB depending on what else is running.

If you use the /3g switch you can get more than 2GB up to about 2700MB but OS performance is penalized.

I'm not sure about 64-bit, I've never tried it but I presume the max file size is then limited only by the amount of physical memory you have.