Intel I219 Experiencing HUGE Package Loss

There are some discussions and seems this issue has been solved. However, the literature is few. So I wrote this note and hope it will help others.

Symptom

Intel Ethernet Connection I219 Series, including I219-V and I219-LM, does not work on Linux, with about 1 Mb/s on speedtest.net and 30-50% ping loss within LAN. This is a problem in the kernel, so both Ubuntu and Fedora will suffer from the same problem. Users from 4.19 to 5.11 all reported this problem. Updating using apt does not help.

Details

• ping from another machine

$ ping -i 0.2 -W 0.2 -c 100 -s 1000 192.168.1.2
100 packets transmitted, 56 received, 44% packet loss, time 20195ms

• device information

# lspci -vvvnn -s 00:1f.6
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (14) I219-V [8086:15fa] (rev 11)
    Subsystem: CLEVO/KAPOK Computer Ethernet Connection (14) I219-V [1558:50e1]
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 129
    Region 0: Memory at 82380000 (32-bit, non-prefetchable) [size=128K]
    Capabilities: [c8] Power Management version 3
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold+)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
    Capabilities: [d0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Address: 00000000fee00338  Data: 0000
    Kernel driver in use: e1000e
    Kernel modules: e1000e
# ethtool -i enp0s31f6
driver: e1000e
version: 5.11.0-40-generic
firmware-version: 0.4-4
expansion-rom-version: 
bus-info: 0000:00:1f.6
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
supports-priv-flags: yes

Solution 1:

Reason

Power management shuts down the cache/memory of the network adapter (See [7] for more information).

Workarounds

• So I need to disable the power management. The work states for Intel devices are called C-State. C-States range from C0 to Cn. C0 indicates an active state (See Intel User Guide/C-State). The first workaround is setting the maximum C-State not very high [3,4].

  vi /etc/default/grub
  # add intel_idle.max_cstate=1 to GRUB_CMDLINE_LINUX_DEFAULT after 
  "quite splash"
  # so that line looks like
  GRUB_CMDLINE_LINUX_DEFAULT="quiet splash intel_idle.max_cstate=1"
  # then save and execute
  update-grub
  # then reboot, you can confirm this is applied by
  cat /proc/cmdline|grep intel
  cat /sys/module/intel_idle/parameters/max_cstate
  

  You can even set max_cstate to 0; someone do this.

• Change a flag about the network adapter's power control [5].

  # on my machine the default value is "auto"
  cat /sys/bus/pci/devices/0000\:00\:16.0/power/control
  echo on > /sys/bus/pci/devices/0000\:00\:16.0/power/control
  # check it is "on" now
  cat /sys/bus/pci/devices/0000\:00\:16.0/power/control
  

Literatures

1. https://bugzilla.redhat.com/show_bug.cgi?id=1652865 This is a report in Jan 2019. he is using 4.29 kernel. No solution was provided.

2. https://bugzilla.kernel.org/show_bug.cgi?id=213651 proposes

   • unload mei* (does not work for me)
   • BIOS settings -> System Management -> Intel AMT Capability, switch it from "Restrict MEBx Access" to "Disabled" (does not work for me -- my BIOS does not have this choice)

3. https://bugzilla.kernel.org/show_bug.cgi?id=213377 is about exactly the same problem. They propose

   • booting the same kernel(s) with "intel_idle.max_cstate=1" (For instructions see How to set intel_idle.max_cstate=1) (almost works -- download speed and package loss is fixed but upload zero)

4. https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1927925

   • They claim the solution in [3] works
   • Comment #93 explains why this bug happens, but too specialized; I cannot fully understand.

5. https://bugs.launchpad.net/ubuntu/+source/linux-oem-5.10/+bug/1930754

   • The trick is to set the boot kernel parameter "pcie_aspm=off" in '/etc/default/grub'
     Like this:
     GRUB_CMDLINE_LINUX_DEFAULT="splash pcie_aspm=off"
     After that run;
     update-grub
     

     (does not work for me)
   • They confirm the solutions in [2] do not work.
   • echo on | sudo tee /sys/bus/pci/devices/0000\:00\:16.0/power/control (almost works -- download speed and package loss is fixed but upload zero)

6. Wired network extremely slow

   • The asker did a lot of work, including reporting that compiling driver ourselves is impossible
   • None of the answers work for me.