Performance Benefits of Adding a Second Network

Solution 1:

Running a separate parallel network is almost definitely not the best way to tackle this. It's quite a bit of work and maintenance, and isn't likely to solve your problem. Keep in mind that it won't automatically balance the traffic over both of your networks. So for example, if all clients use the second network for file sharing, then it will be saturated and directory listings will still be slow. Your first network might still be fast, but nobody will be using it for filesharing.

This is what I would try:

• Measure the network throughput on the fileserver. If it is up near 80% of the theoretical wire speed, then you need a faster network connection. You can use the second ethernet port on your fileserver and bond the two into one network interface with twice the capacity. Your switch has to support this, and there are some limitations to how that additional capacity will actually get used in practice.
• If the network throughput is not very high, then look at other factors of the fileserver to see if they're the bottleneck such as CPU (unlikely), or RAM (more will allow it to cache more).
• The most likely reason is that the disks aren't performing quickly enough. You could look into a better RAID system, use a clustered filesystem, or just shard your data into multiple fileservers. How you do this is very dependent on your applications and OS environment.

If your switch supports SNMP, I would take the time to setup Zenoss or something similar. It will graph the utilization on each port, which will greatly improve your ability to pinpoint the bottleneck. It can also graph the vital statistics of your client and server machines.

Solution 2:

If your problem is network device contention, then moving the transfers to another subnet will help. You'll have to be careful to ensure that the transfers move over the second network and not the first.

However if you are trying to do directory listings on the servers involved in the transfers of those large files, your problems might be because the drives on the computers involved are too busy to step your requests in quickly. In that case, additional network capacity on the server won't help.