Determining why file transfers are so slow
I'm trying to transfer large files (3+ gigs each) across the lan. My connection is 100 megabit to the box, with a gigabit backbone. However, when transfering files between servers and from a server to my local computer, transfer speed hovers around 11 to 13 megabits per second.
Large downloads from the Internet, however, go significantly faster.
I have brought up this problem with my network infrastructure folks, but they insist that everything is fine with the network configuration.
Are there any tools out there that I can install on my computer or the servers to try to isolate the source of the slowdown?
Solution 1:
Firstly check the interface counters on each server, there should be 0 or close to 0 errors reported.
Secondly, check the duplex of both servers. If you are mixing 100mbit and GbE then you may have a duplex mismatch. Ensure that both sides are set to auto/auto, or manually set the speed and the duplex on all interfaces.
Thirdly, what kind of contention is there on the GbE backbone, can you confirm that sufficient headroom exists for your transfer ?
Finally, is your sending server capable of transmitting fast enough. As the comments below suggest, are you limited by the IO bandwidth of the senders drives, or by CPU (if you are using scp or similar) ?
Btw, 11 to 13 mega bytes per second is the theoretical max for 100mbit, are you sure the tool you are using to measure is reporting the correct units?
Solution 2:
Check out the following procedure that I use, which might help narrow down where the problem is.
When I test network throughput, it's best to remove as many variables as possible such as disk i/o or CPU limitations. Here's the procedure I use, which can illustrate the typical max you will ever see in your network.
Tools:
Both these utilities should be available on most modern *nix systems, either preinstalled, via packages, or able to be compiled from source.
- Netcat
- PV
Procedure:
-
Run the following command on one computer to set up the UDP listener. (Using UDP will give higher performance numbers because of the lack of TCP overhead to transfer.)
nc -ulp 5000 > /dev/null -
On the other machine, run the following command, substituting the listener's ip address, to send a stream of zeros across the network to the nc listener. The pv utility will show the current performance as tracked by the receiver.
pv < /dev/zero | nc -u listener-ip-address 5000
Notes:
- The theoretical max on a Gig Ethernet connection is 119MB/s (notice, MB, not Mb)
- TCP can also be tested by removing the -u in both nc commands
- In real world applications, you'll never reach the performance numbers illustrated in the pv output.
This method gives predictable results in a well designed network with properly functioning equipment. For that reason, it is a good tool to help discover if the network itself or the networking equipment is contributing to the perceived "slowness".
Solution 3:
Are you on Windows Vista? Note that Vista throttles LAN transfers any time you have a multimedia app open (including a web browser with flash).
I had this issue with transfers being capped at 10MB/s when they should've been xfering at 30-40MB/s.
You can disable the throttling in a registry key detailed here:
http://www.anandtech.com/systems/showdoc.aspx?i=3233&p=2