Windows file server performance tuning
I seek your opinion and tips/suggestions in optimizing the performance of a new Windows file server I'm building. I've inherited a Dell NF500 storage server (basically Dell 2950 with Windows 2k3 Storage Server OS). It has a PERC 6i with 256mb BBU cache and 6x 750gb SATA drives and 4gb system memory. I think I'm going with RAID6 as I fear loosing a 2nd drive during a long rebuild should the RAID6 volume go bad. The RAID6 volume will use 5x drives with 1x drive as hot spare as well, yes we're super paranoid but are also following our standard that all servers have hot spares.
With that said, I seek your opinion on other tips and suggestions to optimize the performance. It will serve as file server (random R/W and file sizes are typically small but there are some large ones) for both Windows, Macs and Linux clients via SMB/CIFS/NFS.
Any particular settings on the RAID controller side? Currently stripe element set to 256kb (can go up to 512k and maybe 1mb), adaptive read ahead policy and cache write back thanks to BBU. Should I make the stripe element size larger?
Any partitioning/file system level tweaks? I vaguely remember reading something about aligning start of disk partitions, # of drives, building file system with the right block size, etc. Any info, including links, you can send my way so I can review would be very much appreciated.
Any OS level tweaks? Since it's a single RAID volume, does it matter if I put the OS and data store on one partition or should I partition out? I plan to also use VSS, should that be another separate partition? Can it even be on the same partition?
Other best practices?
Thanks in advance. I'm a router/switch/fw guy so this is all a little new to me. C.
Solution 1:
Disk Subsystem: Here's an article from Microsoft re: partition alignment in SQL Server 2008: http://msdn.microsoft.com/en-us/library/dd758814.aspx
The theory explained in the article is why I'm giving you the link, not 'cuz I think you'll be running SQL Server. The workload of a file server is less apt to be as touchy about partition alignment as SQL Server, but every little bit helps.
NTFS:
You can disable last access time stamping in NTFS with:
fsutil behavior set disablelastaccess 1
You can disble short filename creation (if you have no apps that need it) with:
fsutil behavior set disable8dot3 1
Think about the best NTFS cluster size for the kinds of files you're going to be putting on the box. In general, you want to have as large a cluster size as you can get away with, balancing that against wasted space for sub-cluster-sized files. You also want to try and match your cluster size to your RAID stripe size (and, as was said above, have your stripes aligned to your clusters).
There's a theory that most reads are seqential, so the stripe size (which is typically the minimum read of the RAID controller) should be a multiple of the cluster size. That depends on the specific workload of the server and you'd need to measure it to know for sure. I'd keep them the same.
If you're going to have a large number of small files you may want to start with a larger reserve for the NTFS master file table (MFT) to prevent future MFT fragmentation. As well as talking about the fsutil command above, this document describes the "MFT zone" setting: http://technet.microsoft.com/en-us/library/cc785435(WS.10).aspx Basically, you want to reserve as much disk space for the MFT as you think you'll need, based on a predicted number of files you'll have on the volume, to try and prevent MFT fragmentation.
A general guide from Microsoft on NTFS performance optimization is available here: http://technet.microsoft.com/en-us/library/cc767961.aspx It's an old document, but it gives some decent background nonetheless. Don't necessarily try any of the "tech stuff" it says to do, but get concepts out of it.
Layout:
You'll have religious arguments with people re: separating the OS and data. For this particular application, I'd probably pile everything into one partition. Someone will come along and tell you that I'm wrong. You can decide yourself. I see no logical reason to "make work" down the road when the OS partition fills up. Since they're not separate RAID volumes, there's no performance benefit to separating the OS and data into partitions. (It would be a different story if they were different spindles...)
Shadow Copies:
Shadow copy snapshots can be stored in the same volume, or on another volume. I don't have a lot of background on the performance concerns associated with shadow copies, so I'm going to stop there before I say something dumb.
Solution 2:
RAID 6 is going to degrade Read IO quite a bit compared to other RAID options on this setup and it will be very harsh in terms of write IO. Unless you absolutely have to have all of the space I'd suggest you consider RAID 10. You will lose 750GB compared to RAID 6 on your system but the performance difference will make all other performance tweaks you attempt pale into insignificance. Partition alignment, matching stripe sizes to file system block size and changing things like last access time that Evan suggests are excellent suggestions that will improve throughput by 30% or more in total . RAID 10 will be at least 100% faster across the board for Read and Write and its rebuild time is a fraction of RAID 5 or 6's.
RAID 10 - 4 drives with 2 hot spares. Effective Capacity 1500GB. Sustained random read on this will be ~4x the IOPS of your basic drive. Sustained random write will be (up to) 2x the write IOPS of your basic drive unit. Rebuild time - 4-6 hours on a system like yours when idle, maybe double that under average load.
RAID 5 - 5 drives with 1 hot spare. Effective Capacity 3000GB. Sustained random read will be ~4x the IOPS of your basic drive. Sustained random write will ~1x the write IOPS of a single basic drive (assuming 2 reads and 2 writes are required for each write IO). Rebuild time - A day+ when idle, days when under typical load.
RAID 6 - 5 drives with 1 hot spare. Effective Capacity 2250GB. Sustained random read will be ~ 3x the basic drive. Sustained random write will be about 66% the write IOPS of a single basic drive ( 3 reads and 3 writes required for each write IO). Rebuild time - Days when the system is relatively idle, pushing a week when under typical load.
Of course your mileage may vary, if you don't have much sustained random IO then most of this will be hidden by the cache, but the sustained performance hit with RAID 6 is pretty harsh especially when implemented with relatively few drives in the array.
Solution 3:
WSS already does a bunch of the stuff suggested by default: See Windows Storage Server blog
"File-server performance optimizations There are some settings in Windows Server 2003 that speed up network traffic and NAS operations, like removing file-system aliases, turning off 8.3 name creation, and setting the TCP ACK frequency to better utilize the network frame size and speed. See the performance tuning white-paper for more information. OEMs can also improve their performance by setting the interrupt affinity of NIC cards and making sure that proper disk alignment is considered when setting up the RAID array. See below for more information about both of these topics. "
Note that disk alignment should have been done at the factory.