OverlayFS: How Can I Delete Files, Not Hide Them

I am trying to aggregate two mount points in to a single directory so that it will combine the files from both in to a single mountpoint. I am doing some tests with overlayfs and have been able to create the merged mountpoint, but the issue comes when I delete files from this merged directory.

I understand that overlayfs has an upper and a lower filesystem and that if you delete something from the lower that it creates a "whiteout" on the upper filesystem that basically just hides the file from the lower from "shining through" to the merged layer. The file still occupies space on the lower filesystem; it's just not presented as available.

My question is: how do I delete a file from both the upper and lower filesystems and reclaim the space used by the deleted file, and not just hide it? I can manually delete the file from the merged directory and then from the lower directory (is this a bad thing to do?) but df -ha still shows the merged directory as taking up space:

none                            6.8G  1.9G  4.6G  30% /var/www/merged
/dev/loop0                      380M  2.3M  354M   1% /mnt/lower1
/dev/loop1                      380M  2.3M  354M   1% /mnt/lower2
/dev/loop2                      380M  2.3M  354M   1% /mnt/upper
none                            380M  2.3M  354M   1% /mnt/merged

What's the correct way to delete files from an overlayfs filesystem and properly report an accurate disk usage?

Solution 1:

What Overlay does is as follows:

You have 2 directories, lower which is read only and upper which is read write. It's only possible to remove files from the writable layer.

Overlay has not been designed for what you are trying to achieve. Furthermore it expects that the lower dir won't really change, so you might end up with issues.

If you really need to merge multiple drives, use lvm. It also works on files.

Solution 2:

you can not, besides deleting the file using the original path (/lower1/file). OverlayFS is not intended to merge Mounts in that way, to create a joint writable storage.