What hardware combination should I consider for a 20TB file server? [closed]

I want to build a file server for my home media. The server will only be used to serve files to my Media PC. Essentially all it needs to do is be able to expose around 20TB of shared folders.

I don't really have a clue on server hardware, or even if this can be done with desktop hardware. I need some of the following considerations answered please:

1. What motherboards are able to hold 20 SATA connections? I've decided to go for 1TB drives. Obviously I won't be buying these upfront, but add them as my media collection grows. I've never seen a motherboard with 20 SATA connections, so I'm guessing you can get controller cards to handle more HDDs.

2. I've heard of something called a SAN; would it be in my best interests to try and find a cheap second hand SAN, or should I try and find a cheap second hand file server? Alternatively would it be better to try and rig a desktop to perform the task? I mainly want reliability, but price is also a governing factor.

3. Knowing all this there is another solution. I have an old laptop that has USB ports. I could also buy an external USB drive and then have these running all the time connected via USB to this laptop. However, I'm not wild about having 20 USB drives connected to 1 laptop (not even sure if this would work), but the power cables required to power 20 USB drives would be insane!

Any recommendations are most welcome.

Solution 1:

Interesting project. You need to nail down your requirements and your budget, and that will help you nail down what hardware you need for your design.

Step back from the "1TB drive" decision for the moment; a year or two from now that will seem rather limited. Also consider how badly you want the ability to use 20 drives in this system -- that's a lot of power. The number of spindles your plan calls for will impact your case and power supply choices.

1. 20 SATA connections will be tough to find on a single controller, but there are some options. You can get motherboards that provide 6 or 8 SATA ports, then add PCIe SATA controllers for another 6 or 8 SATA ports per card.
   
   The other option is to use a special port multiplier backplane to turn one SATA port into many. Read through the Backblaze pod architecture that Journeyman Geek's answer links to for an example use. They caution that SATA port multiplication can be tricky, and that their solution works well because they use Silicon Image chipsets on both the backplane and the PCIe cards.

2. SAN is one tech design; NAS is another. In a nutshell, NAS provides file sharing; SAN provides block storage (see SAN vs NAS for some of the gory details). From a computer's point of view, NAS provides file shares across (your existing) network, while SAN provides hard disks across (a separate) network. See the difference? The computer can't partition or reformat a NAS share, but SAN storage acts just like local drives.
   
   SAN equipment is typically very expensive because they use enterprise-level components -- 10k or 15k RPM drives, using superfast Fibre Channel or SAS interconnects. Building your own SAN is not for the faint of heart, and probably not a good starter project for someone new to computer hardware.
   
   On the other hand, building a NAS-style server can be done on the cheap, with consumer-grade equipment, and will be a good introduction to system building.

3. Yeah, no. Just don't. USB isn't a real solution here. A USB bus can handle that many devices connected, but performance will crawl if you try to access more than one or two drives at a time. If you want to play around with the laptop and a couple of USB drives, just to try setting up a very basic network file server, feel free -- I guarantee you'll learn something. But that's completely different project than the project you've outlined above.

Solution 2:

Well this is a LITTLE bigger than what you want, but the design/architecture, on the hardware side would be close to what you need. Finding a 20 drive, or even a 15 drive case is gonna be tricky though

Solution 3:

There are several Adaptec, 3Ware and Areca RAID controllers with 24 SATA ports available. Keep in mind that agreggating 20 drives mean you're having 20 times more chance of a catastrophic failure, so RAID-5 or RAID-6 really is mandatory.

You can also add drives to a RAID array with those professional RAID controllers. So my advice is to start with one of these and 8 drives, set up as a RAID6 array, that will provide you about 5.5 TB of effective space. Then grow as you go.

Keep in mind that mixing different drives (even of the same brand) may cause problems and kill performance.

Last but not least, the "desktop drives" have serious vibration interaction when put together in large stacks. Either use the more expensive "enterprise" drives, or use less bigger drives ( 8 x 2TB for instance).

Yes, there definitely have real reasons why storage enclosures, enterprise drives and RAID controllers are expensive. So be prepared to spend some serious money if you don't want to be riddled with troubles.

Solution 4:

1 TB drives are pretty much history when it comes to mass storage (and i mean MASS storage :), 1.5 TB is currently the sweet sport for cost per GB and 2 TB drives aren't too far off either.

Get yourself some rack-mounting enclosures (like the one below, it can hold 5 3.5" HDDs), you can populate them as your collection grows.

if you prefer good looks

and here's a (industrial) Rackmount Chassis for 20x hot-swappable SATA drives ($350):

Here's an excellent illustrated review at WeGotServed.com that you may find helpful.

This solves the problem finding a "motherboard with "20 SATA connections" (the transfer rate up to 6 G will certainly suffice :) and you can connect them to any regular computer you wish to use as your Home/NAS server.

Solution 5:

From Tom's Hardware : 2.5 TB HDD Expected in 2010 (August 7, 2009)

A recent roadmap presented by TDK Corporation--a manufacturer of read/write heads for many hard disk drive (HDD) suppliers--revealed that 640 GB 2.5-inch and 2.5 TB 3.5-inch HDDs should become available around January 2010. For now, Seagate and WD provide HDDs with the largest capacities: 2 TB.

Since you wish to start slow, I would start with one or two 1.5TB USB disks.

Or if you wish, you can get a server with 6-8 internal disk bays, and populate two bays with the 1.5TB disks, then wait. By the time you have need for more, 2.5TB disks will be much cheaper and more available. This is by far the simplest solution and with the best performance.