OpenFiler in large production environment
Does anyone here have any experience with OpenFiler in a large production environment they would like to share? We have about 3 TB of document images and databases and expect to grow rapidly in the near future (perhaps 10 or more TB).
Clearification
We will most likely be connecting to the SAN via ISCSI over GB Ethernet from web, database and FTP servers.
Outside of OpenFiler, your options would be other NAS OSes (FreeNAS, NASLite...), dedicated NAS appliances running custom software or completely rolling your own with a mainstream linux distribution (or Windows if you really want).
I have been experimenting with FreeNAS and OpenFiler for the past month or so. I am putting my eggs in the OpenFiler basket. I haven't been running in a production environment, but all the research I have done points to OpenFiler as being the Enterprise solution compared to other NAS OSes and NAS Appliances. In pretty much all performance reviews I have seen it out performs those solutions. This is of course based on the hardware you are running from and how you tune the server. Also, based on my research, anyone who has purchased the support package has mentioned their support is very exceptional.
You can also see someones comparison of FreeNAS to OpenFiler here to get some ideas of what sort of performance they saw with OpenFiler.
http://www.scribd.com/doc/29643/OpenFiler-vs-FreeNAS
I have not compared OpenFiler to custom solutions though. Some people prefer to support and maintain a more common linux distribution like ubuntu and expose NAS features manually. I think this would be the preferred solution if you are running on uncommon hardware. That is something that I am not interested in doing. I'd imagine you could get as good or possibly better performance with a custom solution depending on your hardware and the support for it.
So as long as you have got a decent hardware setup and a support package, I'd say OpenFiler would be a success in production. Just be sure to check the hardware compatibility page.
Additional Edit: Make sure when you set up openfiler you do it with all new drives which do not have any data. Openfiler uses LVM and doesn't play nicely with drives with existing partitions on them. It will probably be easiest to setup openfiler with nothing on it, then migrate your documents and images later.
Just a couple of quick notes after spending a fair amount of time with Openfiler in a cluster setup and 8Tb of storage:
The 2Tb limit is most likely because you're using MBR (Master Boot Record - Openfiler called it MSDOS, even though that isn't really correct) partitioning, rather than GPT (GUID Partition Table). MBR is limited to 2Tb partitions. See http://en.wikipedia.org/wiki/GUID_Partition_Table
Watch out for Openfiler's iSCSI implementation when using VMWare's ESX/ESXi (or vSphere, which uses ESX/ESXi). Under heavy load, Openfiler's iSCSI module will choke (we've run into this a bunch of times - Openfiler will take volumes offline if it experiences too many errors from the iSCSI module it uses). The Openfiler team is working to convert the existing iSCSI module (EIT) to a different module that is essentially a rewritten version of the existing one, called SCST. A Google search for "Openfiler cmd_abort" will tell you all about the current problems.
If you need e-mails from your RAID controller to report failing disks (and you do), and you don't want to mess around with cramming management stuff into rPath (Openfiler's Linux choice), use a RAID controller that has a LAN port on it, such as a number of the Areca cards.
For a stable, free iSCSI solution, Open-e offers a lite version of their DSS V6, which is fairly similar to Openfiler under the hood, but uses SCST. It's clustering capability isn't as capable as Openfiler's is, but it's far easier to set up, and the management interface is much easier to work with. The catch is that the free version limits you to 2Tb of storage and no cluster capability - you have to pay for a license if you want clustering or more than 2Tb. Their product is VMWare certified, though.
Openfiler's management interface has a few bugs that we encountered. There are situations when creating volumes will result in bizarre sizes that you didn't ask for, and we had an issue when we were exploring the NIC bonding options and had Openfile apply our settings even though we clicked cancel (which resulted in us getting disconnected from the server and having to sort it out from the console via the command line.
If you want speed (who doesn't?), find a way to use multiple NICs and MPIO to multiple your gigabit speeds. If your SAN will be talking mostly to one machine (as is the case if you're using ESX in most cases), Do NOT use link aggregation. It's a common misconception that 802.3ad results in speed - it doesn't unless there are multiple machines pulling data at the same time. If only one machine is accessing the SAN over an aggregated link, you'll only see the speed of ONE of the links (ie: 1Gbps).
Hmm...That wasn't such a quick response after all. :)
No real details (because they didn't run into significant issues as far as I know), but it held up well according to a buddy who ran it for a while on their Windows/Mac network. I think they served a couple dozen terabytes of video and documents out of it.
I have a test server connected to a Promise vTrak 15200 (piece of junk) via iSCSI, and encountered problems using volumes larger than 2TB. I'm actually not sure if this is a limitation of Openfiler or the vTrak - but my solution was to create several arrays on the vTrak and then stripe them together with software raid on the Openfiler.
Other than that - Openfiler seemed to work great (it was our Backup-to-disk storage a month ago when our tape library died). It integrated nicely into Active Directory without much hassle aswell.