How to improve TCP tolerance to out-of-order delivery in Linux balance-rr bonds and/or FreeBSD roundrobin laggs?
Solution 1:
You could try using jumbo frames if your network supports it. It doesn't remove the main problem of triggering TCP out-of-order retransmissions. However, since the ethernet frames are six times bigger, the number of packets decreases which decreases the number of out-of-order events.
Otherwise you should check your use case, do you really need a single TCP connection to get the whole throughput? If there are multiple active TCP connections between the devices, then you should use TCP aware load balancing.