SSD (Intel 530) read/write speed very slow with RAID 10

Solution 1:

While the other answer here beings up some points, your specific issues are due to platform limitations and OS configuration:

• You're throughput-limited by the use of consumer SATA SSDs on an HP Smart Array P410 RAID controller. SATA disks run at 3.0Gbps (3G) on these controllers rather than 6.0Gbps (6G). So that's a barrier that impacts the read speeds of your Intel SSDs; 300MB/s or less per drive.

• The Smart Array P410 controller has specific requirements and best-practices when used with SSDs. In short, the controller is capable of 50,000 IOPS, disable the array accelerator for your SSD volume and performance tops out at ~6 drives.

• Disk performance is not always about sequential read/write speed. Try benchmarking with a proper tool like iozone or bonnie++. You still get the random I/O benefits of your multiple drives.

• At the operating system level, install the tuned-utils package and set the profile to enterprise-performance to remove write barriers from your filesystems and set the right I/O elevator for your setup. This is covered in other questions here, too.

• It looks like you're using LVM. That can have an impact as well...

Here's an iozone report for a G7 ProLiant running with four consumer 6G SATA SSDs (downshifted to 3G speeds) on the same HP Smart Array P410 RAID controller.

You should be seeing ~470MB/s writes and 650MB/s+ reads.

[root@mdmarra /data/tmp]# iozone -t1 -i0 -i1 -i2 -r1m -s56g
        Iozone: Performance Test of File I/O
                Version $Revision: 3.394 $
                Compiled for 64 bit mode.
                Build: linux 

        Record Size 1024 KB
        File size set to 58720256 KB
        Command line used: iozone -t1 -i0 -i1 -i2 -r1m -s56g
        Output is in Kbytes/sec
        Each process writes a 58720256 Kbyte file in 1024 Kbyte records

        Children see throughput for  1 initial writers  =  478209.81 KB/sec
        Children see throughput for  1 rewriters        =  478200.84 KB/sec
        Children see throughput for  1 readers          =  677397.69 KB/sec
        Children see throughput for 1 re-readers        =  679523.88 KB/sec
        Children see throughput for 1 random readers    =  437344.78 KB/sec
        Children see throughput for 1 random writers    =  486254.41 KB/sec

Solution 2:

Oh dear, where to begin?

There is so much involved, and you need a good understanding of everything. Just throwing a bunch of disks against a RAID controller won't yield the results you're looking for.

This can't be easily answered. But at least, here is a list of stuff you have to look at:

• Does the Controller even have the throughput needed? (-> Datasheets)
• Does the Controller even have enough bandwidth to the Host (it does, even at v1.0, since it's x8)
• Does the system's chipset have enough throughput (CPU-Controller)? (unknown)
• What write strategy have you instructed the Controller to employ? (this is what most likely has bitten you)
• Is everything aligned (Partition starts, LVs, PVs)?
• Are the block sizes harmonized? (RAID stripe size, block size, FS blocks,...)
• Is the filesystem optimized to the RAID setup? (Level and Block size)

Since your throughput against the whole RAID (disregarding FS) is significantly lower than a single disk, it's likely you have set up your write strategy wrongly; the Controller is probably waiting for all disks to confirm the write (and unless you've got the RAM on the controller battery backed, this might be in your best interest).