Why bonding does not improve network speed?
I have the following content in /etc/network/interfaces
on one of my server (say server1). On another server (say server2). The content of /etc/network/interfaces
is similar. Except that the mac addresses are changed accordingly and the ip address is changed to 192.168.2.2. Ubuntu 13.10 is installed on both servers.
But when I use iperf -s
on server1 and iperf -c 192.168.2.1
to test the speed. I only get around 10Gbps speed (the speed of a single NIC). My switch is configured as a layer 2 switch. Does anybody know how to make the speed to be 20Gbps through bonding? Thanks.
auto em1
iface em1 inet manual
hwaddress ether c8:1f:66:e2:90:43
bond-master bond0
auto em2
iface em2 inet manual
hwaddress ether c8:1f:66:e2:90:45
bond-master bond0
auto bond0
iface bond0 inet static
address 192.168.2.1
netmask 255.255.255.0
network 192.168.2.0
bond-mode 4
bond-miimon 100
bond-lacp-rate 1
bond-slaves em1 em2
The output of iperf
is the following.
server2:~$ iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 85.3 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.2.2 port 5001 connected with 192.168.2.1 port 34014
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 11.0 GBytes 9.41 Gbits/sec
server1:~$ iperf -c 192.168.2.2
------------------------------------------------------------
Client connecting to 192.168.2.2, TCP port 5001
TCP window size: 22.9 KByte (default)
------------------------------------------------------------
[ 4] local 192.168.2.1 port 34014 connected with 192.168.2.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 4] 0.0-10.0 sec 11.0 GBytes 9.41 Gbits/sec
I also tried to change the mode to balance-alb
and balance-rr
. But they don't lead to improved performance as well. Does anybody know to debug the problem? Thanks.
auto bond0
iface bond0 inet static
address 192.168.2.1
netmask 255.255.255.0
network 192.168.2.0
#bond-mode balance-rr
bond-mode balance-alb
bond-miimon 100
bond-slaves em1 em2
To reach the ~20Gbps transfer bandwidth with LACP bonding (mode 4) one needs to connect the server to more than one client, The maximum transfer speed of one client is capped to the speed of an individual channel (10Gbps /client max. in the above case). LACP "Does not increase the bandwidth for a single conversation"; LACP "Achieves high utilization only when carrying multiple simultaneous conversations" as from the slide 7 from the presentation of the gurus: http://www.ieee802.org/3/hssg/public/apr07/frazier_01_0407.pdf and the same question here: https://serverfault.com/questions/569060/link-aggregation-lacp-802-3ad-max-throughput/569125#569125