Linux's best filesystem to work with 10000's of files without overloading the system I/O

It is known that certain AMD64 Linuxes are subject of being unresponsive under heavy disk I/O (see Gentoo forums: AMD64 system slow/unresponsive during disk access (Part 2)), unfortunately have such one.

I want to put /var/tmp/portage and /usr/portage trees to a separate partition, but what FS to choose for it?

Requirements:

* for journaling, performance is preffered over safe data read/write operations
* optimized to read/write 10000 of small files 

Candidates:

* ext2 without any journaling
* BtrFS

In Phoronix tests, BtrFS had demonstrated a good random access performance (fat better than XFS thereby it may be less CPU-aggressive). However, unpacking operation seems to be faster with XFS there, but it was tested that unpacking kernel tree to XFS makes my system to react slower for 51% disregard of any renice'd processes and/or schedulers.

Why no ReiserFS? Google'd this (q: reiserfs ext2 cpu): 1 Apr 2006 ... Surprisingly, the ReiserFS and the XFS used significantly more CPU to remove file tree (86% and 65%) when other FS used about 15% (Ext3 and ... Is it same now?

Ext4, Btrfs, Reiser4, Reiser3 (with notail option). XFS sucks on metadata intensive operations (like rm'ing lots of files).

You don't really need journalling for those directories (since they can easily be restored), so no matter which filesystem you choose, you can disable journalling.

Back when i used Gentoo, i used ReiserFS 3 for /usr/portage. The reason is simple: it was pretty fast, and it saved a lot of space compared to other filesystems when you are dealing with a huge number of very small files. (using the "notail" option removes this benefit of course) I did notice that over time performance on that filesystem would decrease. This is due to the fact that there are very frequent writes/deletes on /usr/portage and no matter what filesystem you use, after a while fragmentation is gonna make it slower. So sometimes i just took a targz from /usr/portage, reformatted the partition, and unpacked the targz on it again... this gives you a new /usr/portage with no fragmentation.

I don't know if i would still use ReiserFS now.. i would probably go for ext4 (with journalling disabled) or ext2.

About /var/tmp/portage... if you have enough ram, you might want to map that on tmpfs. That way, it will use ram as much as possible, but go to swap when the data does not fit in ram.