How does Linux handle concurrent disk IO?
When a Linux server is serving many concurrent requests to read many different files, does it:
Seek to File_1, read the entire file, then seek to File_2, read the entire file, then seek to File_3, etc etc
Seek to File_1, read part of it (up to the readahead value?), then seek to File_2, read part of it, then seek back to File_1 where it has left off, read more of it, then seek to File_3, etc, etc
If it's the 2nd case, then the server is doing a lot more seeks than is necessary, which would slow things down significantly. In that case is there any tuning I could do?
Solution 1:
In disk I/O there is a thing called elevator. The disk subsytem tries to avoid thrashing the disk head all over the platters. It will re-order I/O requests (when not prohibitted e.g. by a barrier) so that the head will be moving from the inside of the disk to the outside, and back, performing requested I/Os on the way.
Second thing is I/O request merging. If there are many requests within a short time window, which access different portions of the file, the I/O subsystem will try and get all the data in one go, instead of issuing several disjointed requests.
As far as tuning goes. If you are the application writer, there's a lot you can do. You can issue large, sequential I/Os, whenever you can and use fsync() et.al. when you need to be sure that the data is on the platters.
If you are a sysadmin, and you absolutely know, that the data requests of 2 apps leapfrog, and they try to read files sequentially (e.g. you have 2 DVDs being transcoded in parallel), then yes, increasing readahead should help. Otherwise you'd need to take a look at your I/O patterns and sizes, consider your RAID level (if any) and other factors, before doing any tuning. Look at what your real bottlenecks are, before you start tuning, it may be difficult to guess, what's really limiting your system.
Solution 2:
In linux you can define your own scheduling algorithm, you have different possibilities, I had to do a piece on it at school and this article from Red Hat helped me a lot. Although it is specifically for Red Hat you can find these schedulers in virtually any linux distro.