Router limits network performance but its CPU load isn't 100%

I have developed a board using Realtek RTL8211E phy chip and I can assure you, that it is able to operate on gigabit speed :) (actually 10/100/1000). The only problem with this phy chip would be if it had not been connected to the CPU using a gigabit interface (like RGMII for example). I couldn't find the PCB layout of your router on the internet to check it.

However, like I wrote before, it sounds more likely like a duplex mismatch.


It could be something related to the network cards and the path between them and the kernel/cpu (including interrupt processing). You should look at verifying the various "offload" settings (sorry, I'm not familiar enough with FreeBSD to suggest the right tool). Also look for any other network-card driver-specific settings that can be tweaked, and experiment with them .


The cpu is not idling at all, one core 68.7% other 34.3% busy processing interrupts is not idle. The userspace is idle not the kernel.

Not familiar with openbsd, can you set cpu affinity so that one core processes irq259 and the other irq260. Then see how busy each core is.


How about the top's "load average" after some time of doing the speedtest? Does it ever reach 1?

If it is not the CPU, maybe something is wrong with some lower layers? I suggest checking if ethtool or mii-tool show 1000FD in both cases (with and without router-in-the-middle). Maybe your router board forces some link settings and maybe you have a duplex mismatch issue?

Could you run "iperf -s" on your router to check how is the connection between your client and the router?

Regards