Unable to write to disk but disk isn't full

I am using Ubuntu 12.04 and can't write to any file, even as root, or do any other operation that requires writing. Neither can any process that needs to write, so they're all failing. df says I've got plenty of room:

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1       30G   14G   15G  48% /
udev            984M  4.0K  984M   1% /dev
tmpfs           399M  668K  399M   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            997M     0  997M   0% /run/shm

All of the results I find for "can't write to disk" are about legitimately full disks. I don't even know where to start here. The problem appeared out of nowhere this morning.

PHP's last log entry is:

failed: No space left on device (28)

Vim says:

Unable to open (file) for writing

Other applications give similar errors.

After deleting ~1gb just to be sure, the problem remains. I've also rebooted.

df -i says

Filesystem      Inodes   IUsed  IFree IUse% Mounted on
/dev/xvda1     1966080 1966080      0  100% /
udev            251890     378 251512    1% /dev
tmpfs           255153     296 254857    1% /run
none            255153       4 255149    1% /run/lock
none            255153       1 255152    1% /run/shm

Solution 1:

You are out of inodes. It's likely that you have a directory somewhere with many very small files.

Solution 2:

Apparently, the OP has an answer for their particular problem. However, for completeness, the OP's symptoms can also occur if the filesystem has been remounted read only. This has happened to me using a Linux VM whose storage was on a clustered disk system suffering rare intermittent faults. Occasionally, the faults would cause the filesystem(s) to be remounted read only. The eventually observable external symptom was various services becoming nonresponsive as RAM filled (with unflushable disk writes).

At the time, the only resolution was to reboot the system (losing whatever unwritten logs there were). Attempts to remount RW failed. (Unfortunately, I do not recall the error messages returned when attempting these remounts.)

So, ..., not the OP's problem, but someone else arriving on this page may benefit from this information.