C - create file in memory

I'm writing little program in plain C here. What I need is to create a file directly inside memory (not written on hard disk) Currently I can use fopen("filename.txt,"wb") to write to the file.

I know that in linux you can use fmemopen(). Is there a similar solution for win32?

Solution 1:

fdopen is in POSIX.1-1990, and Windows supports it in the form of _fdopen. Using it along with _get_osfhandle and _open_osfhandle, it allows to build up a proper FILE * from a Windows file HANDLE that works with the rest of the C standard library. See also the question Is there a Windows equivalent to fdopen for HANDLEs? for more information regarding this.

Now, the remaining part of the problem is creating a memory-backed Windows file. It turns out there are different approaches to this which approximate this under NT. The most common one seems to be:

1. Creating a temporary file; i.e. non-persistent. In other words, use CreateFile with a path built from GetTempPath and possibly GetTempFileName.

2. ...with the FILE_ATTRIBUTE_TEMPORARY file attribute set [1]; i.e. memory-backed as much as possible.

3. ...and with the FILE_FLAG_DELETE_ON_CLOSE flag set [2]; i.e. hinting that it will never need to be written unless it does not fit in memory.

As long as there is enough memory to keep the file in memory (i.e. low enough memory pressure), the disk will not be used. See this MSDN blogpost from Larry Osterman which describes this approach. Amusingly, he calls such files "temporary" temporary files.

Note that this is not exactly equivalent to fmemopen, which will fail if the file does not fit into the predefined memory area [3] (i.e. fmemopen sets up a pure memory-backed file). However, for many purposes it is close enough (and sometimes the potential backing on disk may be wanted).

In addition, see How to prevent flushing to disk of a memory map opened on a windows temporary delete-on-close file discusses for some discussion regarding these file attributes and other possible approaches.

Finally, for a full example that combines both parts (i.e. creating the memory-backed file and then wrapping it up in a FILE * handle), take a look at the implementations from the fmem library (which was written to provide a cross-platform solution for this problem) or at the one from the libconfuse library.

[1]

Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data. Although it does not directly control data caching in the same way as the previously mentioned flags, the FILE_ATTRIBUTE_TEMPORARY attribute does tell the system to hold as much as possible in the system cache without writing and therefore may be of concern for certain applications.

[2]

The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles.

[3]

Attempts to write more than size bytes to the buffer result in an error.

Solution 2:

You could achieve something close with MapViewOfFile. Win32 API you should use for this are:

CreateFile(...)

CreateFileMapping(...)

MapViewOfFile(...)

Don't forget to unmap and close file mapping once you are done.