How can I tune the initial TCP retransmit timeout?

The initial TCP RTO value of 3s is too long for most LAN-based applications. How can I tune it lower? Is there a sysctl?

See this blog post for how you can make an eBPF program that will override the timeout.

In short, you need to load this sockops program:

#include<linux/bpf.h>
#define SEC(NAME) __attribute__((section(NAME), used))

// TODO: assumes little-endian (x86, amd64)
#define bpf_ntohl(x)  __builtin_bswap32(x)

SEC("sockops")
int bpf_sockmap(struct bpf_sock_ops *skops)
{
  const int op = (int) skops->op;
  if (op == BPF_SOCK_OPS_TIMEOUT_INIT) {
     // TODO: this is in jiffies, and despite `getconf CLK_TCK` return 100, HZ is clearly 250 on my kernel.
     // 5000 / 250 = 20 seconds
     skops->reply = 5000;
     return 1;
  }
  return 0;
}
char _license[] __attribute((section("license"),used)) = "GPL";
int _version SEC("version") = 1;

You can compile and load it with:

clang $CFLAGS -target bpf  -Wall -g -O2 -c set_rto.c -o set_rto.o
sudo bpftool prog load set_rto.o  /sys/fs/bpf/bpf_sockop
sudo bpftool cgroup attach /sys/fs/cgroup/unified/ sock_ops pinned /sys/fs/bpf/set_rto

Nope, you can't; it is hardcoded in the kernel. So change the kernel and recompile.

#define TCP_TIMEOUT_INIT ((unsigned)(3*HZ))     /* RFC 1122 initial RTO value   */

This is what you should get in your include/net/tcp.h.

But I can see someone provided a patch, even though never tried it myself

The initial setting should not affect your overall performance much, as RTO self-adjusts to network conditions. If you do change RTO, you can set it to 1 sec (but no lower).

There is a discussion of this in RFC 1122:

        The following values SHOULD be used to initialize the
        estimation parameters for a new connection:

        (a)  RTT = 0 seconds.

        (b)  RTO = 3 seconds.  (The smoothed variance is to be
             initialized to the value that will result in this RTO).

        The recommended upper and lower bounds on the RTO are known
        to be inadequate on large internets.  The lower bound SHOULD
        be measured in fractions of a second (to accommodate high
        speed LANs) and the upper bound should be 2*MSL, i.e., 240
        seconds.

        DISCUSSION:
             Experience has shown that these initialization values
             are reasonable, and that in any case the Karn and
             Jacobson algorithms make TCP behavior reasonably
             insensitive to the initial parameter choices.

RFC 6298 is a proposed update (published June 2011) that says that RTO can be initialized to a lower value (but no lower than 1 sec), and contains an Appendix containing data that justifies 1 sec as a reasonable initial value.

How can I tune the initial TCP retransmit timeout?

Related

Recent Posts