ext3_dx_add_entry: Directory index full

Background: I'm running lighttpd 1.4.28-1 on CentOS 5.3 to serve the static assets.

Signal: recently, it gets slow sometimes. I'm getting the below errors in the kernel log and /var/log/messages:

proftpd[5145]: (::ffff:xx[::ffff:xx]) - FTP session opened. 
proftpd[5145]: (::ffff:xx[::ffff:xx]) - Preparing to chroot to directory 'xx' 
EXT3-fs warning (device dm-3): ext3_dx_add_entry: Directory index full!
Sep 16 15:30:34 xx last message repeated 489 times

The inode information:

df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/mapper/ddf1_p3
                     77037568 9996012 67041556   13% /
/dev/mapper/ddf1_p1
                       26104      35   26069     1%  /boot
tmpfs                 505834       1  505833     1%  /dev/shm

The contents of the filesystem:

tune2fs 1.39 (29-May-2006)
Filesystem volume name:   /
Last mounted on:          <not available>
Filesystem UUID:          3e9a2bc7-b6d3-4c62-b31d-a9b8ea5ea106
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              77037568
Block count:              77017618
Reserved block count:     3850880
Free blocks:              52289663
Free inodes:              67588146
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1005
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         32768
Inode blocks per group:   1024
Filesystem created:       Fri Oct 30 16:00:47 2009
Last mount time:          Fri Sep  9 01:43:27 2011
Last write time:          Fri Sep  9 01:43:27 2011
Mount count:              7
Maximum mount count:      -1
Last checked:             Fri Oct 30 16:00:47 2009
Check interval:           0 (<none>)
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:       128
Journal inode:            8
First orphan inode:       75267694
Default directory hash:   tea
Directory Hash Seed:      08fb2c9a-2241-4344-973f-045753e3d9ed
Journal backup:           inode blocks

The lighttpd's document root is /data/www/lighttpd, and the quick count talk me the biggest folder only contains 2k files:

find . -type d | while read d; do f=`ls -1 "$d" | wc -l`; echo $f $d; done | sort -n

Is journal corruption? Is there anything else that could cause this?


Solution 1:

I had a similar problem and solved it with fsck after a quick research that led me to this blog post: http://rackerhacker.com/2008/10/13/ext3_dx_add_entry-directory-index-full/

NB: I strongly recommend backing up all your data before doing anything related to the filesystem.