Compressed, read-write file system within a single file?

Is there any free Windows software which can create compressed archives, and subsequently mount them transparently (with write capability) onto another directory?

So far, the only such program I have seen is ImageX (WIM format), but I'm wondering if there is any other alternative that (hopefully) supports a more well-known archive format (e.g. ZIP, 7z, etc.).

Solution 1:

Yes, there is, Pismo File Mount Audit Package http://www.pismotechnic.com/pfm/ap/

you can mount zip iso, etc as folders, and and it has an optional ecrypted container folder

freeware, no nags

Solution 2:

Pismo File Mount Audit Package has a tool to create .pfo files, that files can have internal compression and/or encryption... and are mounted in read/write mode.

Beware it is not perfect, on Windows 10 some apps will not manage well:

• Adobe Acrobat reader give error (path not found or similar) for opening PDF files inside a mounted .pfo as a folder
• TreeSizeFree does not go inside such paths
• Disk manager can not mount VHD / VHDX files inside a .pfo

But Explorer, command line, and also VirtualBox works great.

Having the .vdi inside a compressed .pfo works great and most times with a real great compression ratio (much more than NTFS native compression).

Now, why you can not mount z Zip / 7z in write mode:

• Some can be 'solid' that means to read a file it need to uncompress data till such file data is... decompresion is fast enough to do it with some intelligence algorithms in read mode... but compression would need a lot of minutes.
• Just open such zip / 7z file with its native tool and try to add something, see time it takes

Please do such test on zip / 7z files with a lot of small files, big ones, etc...

It is not imposible to implement such write mode, but it could take as much as a full decompress and compress.

That is the main reason why formats that are not designed for on-the-fly write are not mounted on write mode... the other reason is:

• Files designed for on-the-fly write do not need to be fully recreated
• On the other side, solid files need to be fully re-writed when modify something on inside
• Some file formats do not support to have holes, so any internal file size change causes a realign of all the rest on compressed file

Other reason is security... in terms of not loosing all data...

• When file need to be altered, it would need to be a copy-on-write to be a safe operation... imagine copying a 4GiB just to edit a filename... time would be some minutes

Yes, do not think on having the file on a SSD ultra-fast disk... imagine having such file on a slow, really slow memory card... of less than 10MiB/s ... doing a single edit inside a 4GiB file would took near seven minutes.

One more reason:

• You tell windows to copy one hundred files inside it... windows will process each one as it likes (maybe will not tell the application it is going to copy that list of files, maybe it only tells just one... in real, it is really like that horrible)
• The mounter only see one new file is written, so do the seven minutes process to hold it, but after that windows puts the next one... another seven minutes,... and so on

But wait, it is much worst... maybe windows (or some apps) do not write the whole file in just one order/command... some do it on blocks... so:

• The mounter can not know how much of such write (blocks) will be written and may need such seven minutes for each block

Some apps do this to write/copy a file (the worst case):

1. Open the read file
2. Read a block (1Byte, 512Bytes, 4K, 8MiB, whatever size they are programmed)
3. Closes the read file
4. Open the file to be written or append
5. Write such block
6. Closes the write file
7. Go to step 1 to continue till all blocks are written

So no mounter is able to know how much to 'buffer' for such writes.

Since we are talking of on-the-fly and transparent to the system, the only option is to mount them only in read mode.

There is no technical reason to not do it on write mode, except doing such can lead to a more than a week of time to just copy a few files inside... much less than doing a normal full uncompress, alter, full recompress.

Allways thinking you will have enough free space for holding the final state and previous state at same time (to be safe and ensure not to loose old data).

On the other hand there are file formats designed for write mode:

• They know how to delete (and have gaps)
• They know how to fragment (when adding more data to a file, append) or at least to move the whole file to the end of the container and make a gap where it was before
• Some may know how to re-use enough big gaps (without fragmentation) or fully use gaps (fragmenting)
• Very little of them know how to shrink size or simple do not implement it

Of course, shrink implies fragmentation or a long double move process (to be safe), first to end (if not fit on the gap), then to real position.

Some works with fixed block size, so they can shrink well in background task, it can copy a block to a lower position, making gap going to end of file... and finally reduce file size of such gap size.

That is the theory... i found no tool that do it all... Pismo does not shrink, but does the rest (read/write) with CFS and PFO formats, and for some other formats (read only).

I must investigate such WIM format... but if i do not remember bad it is a M$ format... what about Linux mounting it? Pismo has Windows and Linux versions... and other OS too.

Also Pismo has a SDK, so you can create your own container and 'driver'... or use it inside your app.

Also Pismo is able to mount such a file only for a PID (so only your app can mount it, etc)... and much more... but as i said,... with Windows 10 it has some things not working perfectly well... it is not fully transparent in Windows 10.

Please if someone find a 'perfect' tool, comment and share it, better if is OpenSource.

Solution 3:

To solve Windows 10 issues, mount the pfo with admins rights and with the flag "for all users"... then such apps will see the pfo as a folder.