Root filling up although it should have some space

This has been perplexing me for some time. I'm on Ubuntu 11.04. My root partition is filling up and I can't figure out what's causing it. Even stranger, when I run df -h, this is the output:

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2             9.2G  8.8G     0 100% /

As you see, there should be about 400 MB free, but Avail shows 0. What could be causing this? I've even uninstalled some programs to free the space and it fills up again. Suggestions on how to find out which, if any, files are filling up the space are also welcome.

Solution 1:

A first shot: High traffic in an error log. I guess if you would download videos, you would know it. :)

 sudo du -sh /var/log

• -s is summary
• -h is human readable (k, M, G, T) suffix

For me it is 20M, and beside the log-directory, there is a cache for the installations. Use

 apt-get clean 

If this is your problem.

Of course, you should check /home if it is in your root partition.

If you have to search big files iteratively, I can suggest a series of commands, which can be repeated and is pretty fast after the first initial step:

• You start at a suspicious directory at top level
• Then you search all subdirs and files for their size, and sort numerically:

sudo du -s /var/* | sort -n 
0   /var/crash
4   /var/www
44  /var/games
124 /var/run
2306    /var/tmp
18538   /var/log
251876  /var/cache
1053231 /var/lib

• I stripped the example, to keep it short. So we see, that the bigest subdir is /var/lib, and repeat the step for /var/lib. Now the search is much faster, because the computated results are somehow cached:

sudo du -s /var/lib/* | sort -n 
78116   /var/lib/apt-xapian-index
104580  /var/lib/dpkg
680503  /var/lib/postgresql

and so on. Follow the biggest dirs to find the bigger files. You can't use -sh here, because sorting numerically does not work with the k/M/G/T for kilobyte and so on.

Solution 2:

When formatted, ext2/3/4 filesystem reserves a certain percent of disk space for exclusive use by the root user. By default it's 5%. The purpose of this is that the system can continue working and is able to boot even if non-privileged users fill up all the disk space available to them.

In general, filling up any filesystem beyond 95% of its capacity is not a good idea as this is causing excessive fragmentation which results in performance degradation - even after you free up the disk space.

From man mke2fs:

   -m reserved-blocks-percentage
          Specify the percentage of the filesystem blocks 
          reserved for the super-user.  This avoids fragmenta‐
          tion,  and  allows  root-owned  daemons, such as 
          syslogd(8), to continue to function correctly after
          non-privileged processes are prevented from writing 
          to the filesystem.  The  default  percentage  is 5%.

If you're absolutely desperate, you can use tune2fs to change this value. See man tune2fs for more info