ethernet smp_affinity vs /proc/interrupts vs /sys/class/net/ethX/device
Why are there multiple entries for "eth1" in the form of "eth1-X"?
Because there are multiple tx/rx queues. These queues are often a hash of (local addr, port, remote addr, port) and some other stuff. Suppressing the multiple queues might make it easier to make your application more deterministic, assuming you have few traffic sources. Or you could look up the algorithm and avoid ephemeral ports, if that's easier.
Are you using a realtime kernel? Are you leveraging cgroups
or cpusets
to isolate your application? If you're on a stock distribution kernel, you're leaving a good amount of latency gains on the table. Also, I see 16 CPU-cores. That would indicate that HyperThreading is enabled. How do you know if you're binding to a real versus logical core?