Ubuntu 20.04 Intel Network connectivity issue, bug in igc?

New build on Gigabyte z490 Vision G motherboard. Network link lights lit up, cable directly to the switch. I installed another network card, connected to the same switch, and it's what I'm using for connectivity because it works great (Intel igb driver). Onboard network port will occasionally come up after reboot, but usually shows no link and speed/duplex "Unknown!"

What do you think is going on? What other information do you need?

I don't think it's the cat6 cable, I've tried multiple and multiple switch ports, and cables that are used successfully on the other ports. It's a 2.5Gbps NIC connecting to a 1Gbps switch, I could try picking up a cat7 and see if that helps.

Data requested appended at the end, no /etc/network/interfaces directory, and everything under /etc/network looks stock.

Ubuntu 20.04 server install.

$ ethtool enp5s0 
Settings for enp5s0:
    Supported ports: [ ]
    Supported link modes:   10baseT/Half 10baseT/Full 
                            100baseT/Half 100baseT/Full 
                            1000baseT/Full 
                            2500baseT/Full 
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full 
                            100baseT/Half 100baseT/Full 
                            1000baseT/Full 
                            2500baseT/Full 
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: Unknown!
    Duplex: Unknown! (255)
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: off (auto)
Cannot get wake-on-lan settings: Operation not permitted
    Current message level: 0x00000007 (7)
                   drv probe link
    Link detected: no

$ lspci -knn
5:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15f3] (rev 02)
    Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
    Kernel driver in use: igc
    Kernel modules: igc

$ dmesg | grep igc
[    3.181094] igc 0000:05:00.0: 4.000 Gb/s available PCIe bandwidth (5 GT/s x1 link)
[    3.181096] igc 0000:05:00.0 eth0: MAC: 18:c0:4d:00:00:00
[    3.181657] igc 0000:05:00.0 enp5s0: renamed from eth0

           *-network
                description: Ethernet interface
                product: Intel Corporation
                vendor: Intel Corporation
                physical id: 0
                bus info: pci@0000:05:00.0
                logical name: enp5s0
                version: 02
                serial: 18:c0:4d:00:00:00
                capacity: 1Gbit/s
                width: 32 bits
                clock: 33MHz
                capabilities: bus_master cap_list rom ethernet physical 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
                configuration: autonegotiation=on broadcast=yes driver=igc driverversion=0.0.1-k latency=0 link=no multicast=yes port=twisted pair slave=yes
                resources: irq:17 memory:51a00000-51afffff memory:51b00000-51b03fff memory:51900000-519fffff

Requested data:

$ cat /etc/netplan/*.yaml
network:
  ethernets:
    enp6s0f0:
      dhcp4: no
    enp6s0f1:
      dhcp4: no
    enp5s0:
      dhcp4: no
      optional: true
  bonds:
    bond0:
      interfaces: [enp6s0f0, enp6s0f1, enp5s0]
      addresses: [192.168.37.10/24]
      gateway4: 192.168.37.1
      nameservers:
        addresses: [192.168.37.1]
        search: [home.lan]
      parameters:
        mode: balance-alb
  version: 2

$ sudo lshw -C network 
  *-network                 
       description: Ethernet interface
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:05:00.0
       logical name: enp5s0
       version: 02
       serial: 18:c0:4d:00:00:00
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list rom ethernet physical 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igc driverversion=0.0.1-k latency=0 link=no multicast=yes port=twisted pair slave=yes
       resources: irq:17 memory:51a00000-51afffff memory:51b00000-51b03fff memory:51900000-519fffff
  *-network:0
       description: Ethernet interface
       product: 82576 Gigabit Network Connection
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:06:00.0
       logical name: enp6s0f0
       version: 01
       serial: 00:1b:21:00:00:00
       size: 1Gbit/s
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igb driverversion=5.6.0-k duplex=full firmware=1.5.1 latency=0 link=yes multicast=yes port=twisted pair slave=yes speed=1Gbit/s
       resources: irq:16 memory:51820000-5183ffff memory:51400000-517fffff ioport:3020(size=32) memory:51844000-51847fff memory:51848000-51867fff memory:51868000-51887fff
  *-network:1
       description: Ethernet interface
       product: 82576 Gigabit Network Connection
       vendor: Intel Corporation
       physical id: 0.1
       bus info: pci@0000:06:00.1
       logical name: enp6s0f1
       version: 01
       serial: f6:3b:fc:00:00:00
       size: 1Gbit/s
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igb driverversion=5.6.0-k duplex=full firmware=1.5.1 latency=0 link=yes multicast=yes port=twisted pair slave=yes speed=1Gbit/s
       resources: irq:17 memory:51800000-5181ffff memory:51000000-513fffff ioport:3000(size=32) memory:51840000-51843fff memory:51888000-518a7fff memory:518a8000-518c7fff

I'll answer my own question, this is an intel firmware issue. Hope this helps someone else.

Gigabyte has a NIC firmware posted on their website which is windows-only, https://download.gigabyte.com/FileList/Driver/mb_driver_intel-i225-firmware-tool.zip. Intel has Linux NVMUpdate utilities on their site, but not packaged with this firmware, as far as I could tell. Available at https://downloadcenter.intel.com/download/22283/Intel-Ethernet-Adapter-Complete-Driver-Pack

So, download both, copy FXVL_15F3_V_2MB_1.45.bin from the windows updater to the linux location, in my case:

cp ~/Nvmupdate145/i225/FXVL_15F3_V_2MB_1.45.bin ~/NVMUpdatePackage/700_Series/700Series/Linux_x64

Add this to nvmupdate.cfg:

BEGIN DEVICE
    DEVICENAME: NNT
    VENDOR:     8086
    DEVICE:     15F3
    SUBVENDOR:  1458
    SUBDEVICE:  E000
    NVM IMAGE:  FXVL_15F3_V_2MB_1.45.bin
    IMAGE DOWNGRADE: TRUE
END DEVICE

And run the utility. You should see:

$ sudo ./nvmupdate64e 

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.35.23.3
Copyright (C) 2013 - 2020 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [*****.....]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Gigabit ET Dual Port         1.5(1.5)   10C9 00:002 Update not    
    Server Adapter                                               available
02) Intel(R) Ethernet Controller (2)    1.69(1.45)   15F3 00:006 Update        
    I225-V                                                       available

Options: Adapter Index List (comma-separated), [A]ll, e[X]it
Enter selection: a
Would you like to back up the NVM images? [Y]es/[N]o: n
Update in progress. This operation may take several minutes.
[********+.]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Gigabit ET Dual Port         1.5(1.5)   10C9 00:002 Update not    
    Server Adapter                                               available
02) Intel(R) Ethernet Controller (2)    1.69(1.45)   15F3 00:006 Update        
    I225-V                                                       successful


Tool execution completed with the following status: All operations completed successfully.
Press any key to exit.

And, didn't even need a reboot:

$ ethtool enp6s0 
Settings for enp6s0:
    Supported ports: [ ]
    Supported link modes:   10baseT/Half 10baseT/Full 
                            100baseT/Half 100baseT/Full 
                            1000baseT/Full 
                            2500baseT/Full 
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full 
                            100baseT/Half 100baseT/Full 
                            1000baseT/Full 
                            2500baseT/Full 
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: off (auto)
Cannot get wake-on-lan settings: Operation not permitted
    Current message level: 0x00000007 (7)
                   drv probe link
    Link detected: yes

Based on the inspiration from the answers above, for my Ubuntu 20.04.1 system running on an Asus WS X299 SAGE II motherboard with the Intel I225-LM chipset, I switched to the 5.8.7 mainline generic kernel (the latest 5.8 line at the time of this writing) using the Kernel PPA builds. (steps inspired by the article here).

1. You’ll need to download the following files from here:

   • linux-headers-*all.deb
   • linux-headers-*generic*.deb
   • linux-image-unsigned-*generic*.deb
   • linux-modules-*generic*.deb

2. Once they’ve been downloaded, run sudo dpkg -i deb linux*.deb

3. Reboot! (e.g. sudo reboot)

4. Once your system is back up and running, you can verify you’re on kernel 5.8.7 by running uname -sr

Doing this fixed the issue I was having with my built-in NICs on the and the entire system has been stable without issues under high memory and CPU pressure for several days continually.

I initially tried going the firmware update route but got stymied as I have an I225-LM rather than the I225-V. The latter seems to have updated firmware that can be brought into a Linux environment while I couldn’t find a similar outcome for the former.

John, thank you so much for posting after you figured it out for yourself. I just completed a new build with a Gigabyte Z490i and was totally flummoxed when the onboard LAN worked, and then didn't, and then did, and then didn't.

As I write this I'm about to try your method, but first I'm verifying things look similar:

% ethtool enp4s0
Settings for enp4s0:
    Supported ports: [ ]
    Supported link modes:   10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
                            2500baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Supported FEC modes: Not reported
    Advertised link modes:  10baseT/Half 10baseT/Full
                            100baseT/Half 100baseT/Full
                            1000baseT/Full
                            2500baseT/Full
    Advertised pause frame use: Symmetric
    Advertised auto-negotiation: Yes
    Advertised FEC modes: Not reported
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 0
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: off (auto)
Cannot get wake-on-lan settings: Operation not permitted
    Current message level: 0x00000007 (7)
                   drv probe link
    Link detected: yes

% lspci -knn | grep -i -C3 ethernet
04:00.0 Ethernet controller [0200]: Intel Corporation Device [8086:15f3] (rev 01)
    Subsystem: Gigabyte Technology Co., Ltd Device [1458:e000]
    Kernel driver in use: igc
    Kernel modules: igc

[    3.572001] igc 0000:04:00.0: 4.000 Gb/s available PCIe bandwidth (5 GT/s x1 link)
[    3.572005] igc 0000:04:00.0 eth0: MAC: [redacted]
[    3.581425] igc 0000:04:00.0 enp4s0: renamed from eth0
[   14.359088] igc 0000:04:00.0 enp4s0: igc: enp4s0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX

The first hurdle was that https://download.gigabyte.com/FileList/Driver/mb_driver_intel-i225-firmware-tool.zip was some weird zip format that I couldn't extract with anything except a Windows computer. So I did that, copied the FXVL_15F3_V_2MB_1.45.bin file to the server, and proceeded with your instructions.

% sudo ./nvmupdate64e

Intel(R) Ethernet NVM Update Tool
NVMUpdate version 1.35.33.4
Copyright (C) 2013 - 2020 Intel Corporation.


WARNING: To avoid damage to your device, do not stop the update or reboot or power off the system during this update.
Inventory in progress. Please wait [**|.......]


Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Controller        1.56(1.38)   15F3 00:004 Update
    I225-V                                                       available

Options: Adapter Index List (comma-separated), [A]ll, e[X]it
Enter selection: 1
Would you like to back up the NVM images? [Y]es/[N]o: y

And the after:

Num Description                          Ver.(hex)  DevId S:B    Status
=== ================================== ============ ===== ====== ==============
01) Intel(R) Ethernet Controller        1.69(1.45)   15F3 00:004 Update not
    I225-V                                                       available

Unfortunately, this didn't seem to help my problem.

After a reboot, the link was still dead, like there wasn't even a cable plugged in. Currently the only way I've found to get it to work is to boot into the Ubuntu live installer, get to the point where I can configure the network, get it an IP via DHCP, then reboot into my on-disk install, and it works until the next reboot. Obviously this isn't ideal.

I was able to solve this thanks to your workaround.

I had a different motherboard, so I downloaded the Windows LAN Driver from Asus's website at
https://www.asus.com/Motherboards/ROG-STRIX-Z490-E-GAMING/HelpDesk_Download/.

I then downloaded the NVM Update Utility at
https://downloadcenter.intel.com/download/24769 since it has a Linux version.

The name of the file should be similar to FXVL_15F3_ASUS.bin, but the ending will be different depending on the various Z490 motherboard manufacturers. This can be found on the driver website for the specific motherboard you are using.

After unzipping this file, I was left with two zip files,

• I225V_End_User_FW_UPDATE.zip
• I225V_Intel_Gigabit_Ethernet_Driver_V1.0.1.4_WIN10_64-bit.zip.

1. I found the FXVL_15F3_ASUS.bin in I225V_End_User_FW_UPDATE.zip and copy-pasted this file over into the /700Series/Linux_x64/ directory for the NVM Update Utility.

2. Next, we need to find the config file for the i225 driver. After unzipping and searching the Asus LAN Driver files, I found the proper config settings at config_i225.txt in I225V_End_User_FW_UPDATE.zip. Again, the location of the config_i225.txt might vary depending on the motherboard manufacturer's LAN driver.

3. Lastly, I navigated to nvmupdate.cfg in /700Series/Linux_x64/, and added this to the end of the file:

   BEGIN DEVICE
       DEVICENAME: NNT
       VENDOR:     8086
       DEVICE:     15F3
       SUBVENDOR:  1043
       SUBDEVICE:  87D2
       NVM IMAGE:  FXVL_15F3_ASUS.bin
       IMAGE DOWNGRADE: TRUE
   END DEVICE
   

4. Once this is added, type sudo ./nvmupdate64e. After following the on-screen instructions, Ethernet should connect immediately. Hope this helps anyone experiencing this issue!

Option 1: Switch to Pop!_OS Kernel

I have the same NIC (I225-V) on an Asus B550 motherboard (ROG-STRIX-B550-I). I got the NIC working by (accidentally) switching to the latest Pop!_OS 5.4 kernel.

1. Add the Pop repository:

sudo add-apt-repository ppa:system76/pop

2. Do an apt update.

3. And when you do an apt upgrade, it should recommend to upgrade the kernel to the Pop version, something like this 5.4.0-76XX-generic.

The 76 signifies that it's Pop's custom kernel, and the next two numbers provide the Debian version of the kernel.

Right now, I'm running kernel 5.4.0-7634-generic with Xubuntu. The NIC comes up and registers at 1Gbit/s.

Option 2: Out-of-tree module compilation using the Pop!_OS source

If you don't want to go so far as switching kernels, then you can compile the igc driver from the Pop source using the Ubuntu kernel build as your base library.

1. Download the latest source from Pop's github repository:

wget https://github.com/pop-os/linux/archive/master_focal.zip

2. Decompress the archive:

unzip master_focal.zip

3. Change to the driver source directory:

cd linux-master_focal/drivers/net/ethernet/intel/igc

4. Compile the driver using your current 5.4 kernel build directory:

make -C /lib/modules/`uname -r`/build M=$PWD

For example, I compiled the driver using the 5.4.0-42-generic kernel source files:

make -C /lib/modules/5.4.0-42-generic/build M=$PWD

Substitute 5.4.0-42-generic with uname -r going forward.

You can inspect that everything worked with a modinfo:

modinfo igc.ko

The vermagic in the output should match your current kernel, like this:

vermagic:       5.4.0-42-generic SMP mod_unload

5. Now install the driver object file. I opted to do this manually so that I could make a backup of the Ubuntu driver in case things went south with the Pop driver.

sudo cp -vp igc.ko /lib/modules/5.4.0-42-generic/kernel/drivers/net/ethernet/intel/igc/igc.ko-latest-pop
cd /lib/modules/5.4.0-42-generic/kernel/drivers/net/ethernet/intel/igc/
sudo cp -pv igc.ko igc.ko-5.4.0-42-generic-original
sudo cp -pv igc.ko-latest-pop igc.ko

6. Reboot and enjoy a NIC that works using the latest driver from Pop!

After booting into the kernel with the updated driver, you can check that you are using the compiled driver using modinfo:

modinfo igc

The filename should match the location of the driver you installed, for example:

filename:       /lib/modules/5.4.0-42-generic/kernel/drivers/net/ethernet/intel/igc/igc.ko

WARNING YMMV. If you choose to compile the Pop drivers, make sure you have a working backup kernel, and that you are comfortable with switching kernels in grub in case you bork the kernel while installing the driver. I edited my grub defaults (in /etc/default/grub) to be able to easily switch kernels.

GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=9

Beware: There are still issues with the latest driver.

I have noticed that even this version of the igc driver still has bugs in kernel 5.4. For instance, the /proc/net/dev stats are blank and do not update under the igc driver for the NIC. The stats from ethtool -S are updating. I'm not sure if they are accurate or not.

Experimentation

I've compiled and installed the igc driver from linux-5.6.19 kernel source. It also seems to be working under the Pop 5.4 kernel, but the 5.6 driver version still has the /proc/net/dev stats issue.

I also tried compiling the latest source for the igc driver in the 5.7 and 5.8 kernels. And it is a no go if you are using the 5.4 source. There are dependency issues under 5.7 and 5.8.

It looks like the 5.8 kernel adds some power management dependencies that are lacking in 5.4. I get the following error:

error: ‘DPM_FLAG_NO_DIRECT_COMPLETE’ undeclared

when I run:

make -C /lib/modules/5.4.0-7634-generic/build M=$PWD

using the latest source as of 8/19/2020:

svn export https://github.com/torvalds/linux.git/trunk/drivers/net/ethernet/intel/igc