Why am I seeing such low SMB transfer throughput?

Solution 1:

Your performance gap matches with a common experience when Samba (not sure if this is still the default; it was for a long time) is configured with the default read and write socket buffer size of 1024 bytes.

I used to see this frequently with Linux and Mac machines. Hopefully it's not still that case.

There is a socket option argument in samba's configuration file where you can set the read and write socket buffer size. Suggest you set both to 8192 bytes (8 KiB). 4 or 8 KB is often similar, but I haven't tested that on a gigabit link.

Also, don't expect a single TCP connection to benefit from a bonded link, the traffic will almost always go through one of the links; otherwise you end up with a lot of out-of-order packets to deal with; so only expect a load-balancing benefit when servicing multiple clients. Even then, you should look up the different bonding modes, and know that for at least "mode 4" (IEEE 802.3ad) bonding, there are basically two transmit hash modes, which determine which slave interface to send out on. There is layer-2 hashing (default) and layer-3 hashing. If sending the bulk of your data via gateway, the layer-2 hash will not distribute well, as the gateway's MAC address will be the same. Consider using layer-3 instead.