Fixing bufferbloat in home network

In our home network, with continuous network IO happening latencies get ridiculous. It's fine with light loads but quickly becomes unusable if I'm, say, rsyncing a large number of small files, --other transfers essentially stop. Pings get through just barely.

Simple topology -- a broadband modem / WIFI router in a single box, with some wireless and some wired clients.

I've tried setting ifconfig wlan0/eth0 txqueuelen 1 on all clients, this seems to help a bit but not by much.

Tips welcome: how should I go about diagnosing and eliminating latency problems? Are there more configuration settings I can set on clients, or maybe a better WIFI routers could help?

Solution 1:

What you need is Quality of Service and a tweakable Queuing Algorithm. I'd recommend a setup something like this if you are happy to play with the internals of Linux / FreeBSD and have some spare cash:

+------+     +--------+      +------+<----> Wireless Access Point
|Router|<--->|Firewall|<---->|Switch|
+------+     +--------+      +------+<----> Wired computers

Where the Firewall is a Linux or FreeBSD or similar machine with 2 network cards which can impose some advanced queuing and Quality of Service rules on the traffic.

I had to do something similar to stop DNS lookups timing out when doing heavy data transfers over a slower internet connection.

The other option is to get a better router - one that you can install dd-wrt or open-wrt on as I believe these two firmwares provide more in the way of QoS etc than bog-standard home routers.

Solution 2:

I've tried setting ifconfig wlan0/eth0 txqueuelen 1 on all clients...

That command reduces the buffer size to 1 packet on that machine. The important machine to run this command on is the bottleneck for your network - that is, the machine that has the gigabit connection on one side, and the much slower connection to the Internet on the other. Thus, it's the router you want to run that command on, not the clients!
Make sure you run the command on the right interface (just the outbound LAN interface).

This will of course only work if your router runs Linux, and allows you to run commands. Fortunately, most custom firmwares for home routers (DD-WRT, OpenWRT, Tomato) do.

It should be noted that reducing the buffer size like this can screw up the QoS, so if you're using the QoS on your router, you may be better off leaving the buffer size alone and just using QoS to cap the upload/download speeds of every client to be significantly below the connection's max upload/download... though this has the obvious disadvantage of having unused bandwidth when only one person is connected.

Solution 3:

QoS or Quality of Service is some kinda ordering and setting priorities to the packets going to be processed by the router ... This is what exactly should care for when you have such problems... Try searching in your routers configuration panel and enable QoS... It's better to use the manual settings in your case but you should know the priority order of the packets you wanna be processed first to last .