Upgrading RAID controller to 4kn - do I have to reinitialize the array to use 4kn?
I have an enterprise-grade workstation with an Adaptec 6405E RAID controller in it. According to Adaptec, this RAID controller does not, and will never be able to, natively support 4K disk sectors. If you stick a 4K disk in it, then if it has 512-byte sector emulation mode (512e), it will use 512e. If it doesn't have 512e, the disk just won't work at all.
Connected to my 6405E are four HGST SATA disks that all have 4K native sectors, but they do support 512e. The disks are in RAID10 and the array is working "reasonably well" (performance is fine to good, but not amazing).
Without wiping the data on the disks and re-initializing the array, assuming I plug in an Adaptec 71605E, which does support native 4K sectors, will the controller use 4K sectors when interfacing with my disks? Or is this decision of using 512e or 4Kn baked into the structure of the on-disk format such that I'd have to wipe the disks to do that?
This question is just about whether I have to backup my data and re-initialize the array, or whether the controller can (automatically, or with manual intervention) be asked to "switch over" to Advanced Format 4Kn addressing. I already know for certain that if I did wipe and reinitialize the array, I could definitely set it up from scratch to use 4Kn on all the drives, using this new RAID controller.
Note that I am already quite familiar with the arcconf command line utility, and have previously used it to upgrade this array from RAID0 to RAID10 (yes, I know, I should've never been using RAID0 to begin with, but I got lucky, okay?). If there is some feature of the arcconf utility to "switch over" from 512e to 4Kn on the Adaptec 7-series controllers, I'd love to know about it, so I can use that to avoid having to reformat and temporarily offload the data to a backup location.
In the worst case, I have off-site backups of critical data already, but the system has so much software loaded on it that it would be cheaper (in terms of time spent) for me to do a block-layer copy of the entire array onto another disk -- probably a cheap 4TB disk connected to the mobo via AHCI -- then copy it back over once the logical array is reinitialized. Compared to the prospect of reinstalling everything (a metric ton of proprietary Windows programs with activation and such), that'd actually be cheaper and faster.
Your disks are either 512e (512 sectors on SAS/SATA interface) or 4k native (4k sectors on SAS/SATA interface), and unfortunately there is no way to change that via software or jumpers etc. You select the transfer mode when you buy the disks. Buy 4k native disk if you have adapter that supports 4k native on interface.
Update: and, once again, the disks never "fallback from 4kn to 512e" etc. The disks are either 512e - it means they will always send data in 512 sized sectors over SAS/SATA interface, or 4kn, it means disks will always send data in 4k sized data over SAS/SATA interface, and it depends only on disk, not on RAID adapter capabilities. The difference between 512n and 512e is that on the physical media sectors are sized as 512 for 512n, and 4k for 512e (disk chip translates each 4k sector on plates into 8 x 512 sectors on interface), on the interface 512e disk will always transmit only sectors at 512 bytes, no matter what adapter it is connected. The part number differ for 512e and 4kn disks, for example:
ST6000NM0014 - 6TB SAS drive with 4k sectors on SAS interface (called 4kn drive);
ST6000NM0034 - 6TB SAS drive with 512 byte sectors on SAS interface (called 512e drive)
both of these have 4k sectors on disk media, so care must be taken about sector write alignment in 512e case.
And you can still buy 512n disks, for example: ST4000NM0023 - 4TB SAS drive with 512 byte sectors on interface and 512 sectors on media, so no need to care about sector alignment for this drive.
The RAID adapters fall intro 3 categories: a) the oldest ones that do not know about 4k sectors - they work with 512n and 512e disks, however issues may arise with write performance if writes are not aligned by 8xsector boundaries on 512e drives, b) the not so old ones that know about 4k internal sectors and about 512e emulation, but only work with 512 sectors on interface - less problems with alignment as controller cares about that, c) the very new ones that are able to work with 4k sectors on interface. Only these will work with new 4kn disks that pass native 4kb sector as 4kb sector onto SAS/SATA interface.
Also, only Windows 8, 8.1 or later OS support 4kn drives (for server, 2012 or later version). Majority of old utilities that directly work with disks will NOT work properly with 4k sectors as they assume sectors are always 512 byte sized. instead of checking.
So, to avoid any confusion with alignment and get the maximum performance, use new 4kn drives, new 4kn enabled adapters, and new OS.
I think this statement below is not correct: "certain 4kb-native disks may choose to support 512-byte emulation. If they support 512-byte emulation, they can switch between this mode and 4kn depending on what the disk controller supports; they'll prefer 4kn, but fall back to 512e if they have to".
Sector size is fixed in the factory. I am not aware about any drive that is able to automatically change sector size on interface depending on RAID adapter capabilities. What I see in Seagate order systems are very separate part numbers depending on sector size on interface. Impossible to change sector size after disk is ordered (could be possible by some hacking, changing disk firmware, etc. but not officially supported). So if your drive is 512e it will always send only 512 byte sectors on interface, and never 4k sectors. If your drive is 4kn, it will always send only 4k sectors on interface and never 512 sectors. You decide only when ordering, as its different part numbers.
The possible drive formats are (the number indicates sector size on interface):
512n - 512 on disk, 512 on interface (simple)
512e - 4k on disk, 512 on interface (performance complications possible on old systems)
4kn - 4k on disk, 4k on interface (simple, best interface performance, do not work on old systems)
n or e means if specified sector size on interface is native disk sector size (n), or emulated size (e).
And the answer is: your disks are 512e disks (as they work with adapter that does not support 4kn disks), they are not 4kn disks. Your 512e disks will never use 4k sectors on interface with any RAID adapter. BTW, only the very new 6TB drives from Seagate are possible in 4kn format, and new 6TB and 8TB from HGST also can be ordered as either 512e or 4kn. All drives up to 4TB before were only available in 512e or 512n, I was not able to purchase any 4kn drive for testing before this September.
My personal recommendation is to use LSI adapters. Most compatible with best error reporting from anything I tested, and best performance. With latest firmware release, fully support 4kn disks. I am using many Smart Array adapters from HP also, as they come with HP ProLiant servers, but still no information if and when SmartArray adapters will support 4kn disks. Only host bus adapters mentioned in release notes - very recent firmware update enables support for 4kn disks. So, still 4kn disks are very new.
Hope I helped to make it clear.
Thank you for this great post which I have found by accident searching for the most reliable hdd solution for VMware environment and understanding the differences between 512n, 512e and 4kn.
VMware suggests to still use 512n disks as they have the highest performance due to no need for emulation and the fact that 4kn is not supported even by VMware ESXi 6.0 yet: http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2091600
Conclusion: For a maximum performance on VMware ESXi systems using direct attached storage I recommend to use 512 native formatted hard drives. I will first start using 4kn disks when my adaptec raid controllers as well the operating systems are capable to communicate with 4kn without any emulation.
Cheers, Daniel