linux - disk space missing

# df -h /
Filesystem            Size  Used Avail Use% Mounted on
rootfs                9.9G  7.2G  2.2G  77% /

# du -hx --max-depth=0 /
3.2G    /

As you can see, df says 7.2GB is used, but du can only find 3.2GB of it. The server has been rebooted since I noticed this, so it's not a deleted file. Additionally, lsof doesn't show me anything interesting. What else could it be?

Solution 1:

I was having the exact same issue on my ext4 system and just wanted to post my solution for future reference. When my drive initially filled out, I deleted a bunch of logs out of /var/log. That cleared up a couple GB, but within a few days I ran out of space again, and du -h and "mount --bind / /mnt" did not point to the culprit. What finally got it was when I ran lsof.

lsof
...
rsyslogd   1766      root    2w      REG                9,1   12672375940     264014 /var/log/messages (deleted)
...

When I had deleted the messages log file, the rsyslog service still kept it open, but hidden. Running a "touch /var/log/messages;service rsyslog restart" cleared up the problem and my disk space was reclaimed.

The lsof output can be a bit overwhelming, especially if you have a busy system (it was over 1000 lines long on mine). If you grep for "deleted" in the lsof output, it should help to pinpoint the problem process.

Solution 2:

Since you're using the -x option, I assume you have other filesystems mounted? It could be that you have another partition mounted on top of a directory that wasn't empty.

Solution 3:

Make sure all folders "behind" your mountpoints are empty. Out of experience i'd say most probably you hide some data behind a mountpoint.

you can check what is in the folder behind the mountpoint without having to unmount the disk/partitions (which can be nice if for example you would have to unmount /usr). Just do a

# mount --bind / /mnt
# du -shx /mnt

also as a side note:

# du -shx /

does the same as your du... but is quite a bit shorter (-s stands for summarize)