Linux Read-Ahead Downsides

Are there any downsides to raising the size of the read-ahead cache?

On our farm, we're currently running at 256, and upon raising that higher, we are seeing significant throughput gains.

[root@server~]# hdparm -tT /dev/sda /dev/sda: 
Timing cached reads: 7352 MB in 2.00 seconds = 3677.62 MB/sec 3
Timing buffered disk reads: 244 MB in 3.10 seconds = 78.68 MB/sec 

[root@server ~]# blockdev --setra 10240 /dev/sda 

[root@server ~]# hdparm -tT /dev/sda /dev/sda: 
Timing cached reads: 11452 MB in 2.00 seconds = 5728.52 MB/sec 
Timing buffered disk reads: 422 MB in 3.17 seconds = 133.04 MB/sec

We are running on 2.6.

The downsides are

• Wasted IO. Your computer may spend time reading things from the drive it doesn't need
• Wasted memory. The stuff it read that didn't need to be read is stored in memory that could have been used for something else.

You probably need to come up with a test that is more representative of your real-world usage then hdparm -t. hdparm -t does sequential access which is case where a large read-ahead will be very helpful. When you have lots of random access the gains may not be nearly as useful and probably harmful.

As far as I know there is no danger to your data, just a potential for you performance to get worse depending on your specific work load. This is probably one of the types of things that you are going to have to tweak and test. I don't think I would make such a large change (from 256-10240) in a single step though.