NetworkManager timeout and "ip-config-unavailable" on Ethernet
The probable cause was a defective Ethernet port on the router. I didn't find the evidence to support this until 30 April 2015, and I didn't make the connection until 10 August 2015.
What I Should Have Done
A few additional troubleshooting steps:
-
I should have run
sudo ethtool -S eth0
to see if there were any errors in the network interface controller (NIC) statistics.Since I was able to send but not receive over Ethernet, I might have seen something like the
rx_crc_errors
statistic increment on each receive attempt. -
I should have tried plugging in the router end of the Ethernet cable into a different router port.
Sure, I plugged in the computer end of the Ethernet cable into two different computers, but that led me to believe that a computer (Craptop) was faulty because the other (Deltique) was able to connect to the router.
-
On the working computer, Deltique, I should have run
sudo ethtool eth0
to see if a good link was established.Deltique has a 1Gbps capacity, and so does the router, so
ethtool
should have shownSpeed: 1000Mb/s
andDuplex: Full
. I would later discover on 30 April 2015 that the defective port would have causedethtool
to return different, more concerning values. -
I should have run a speed benchmark to test the Ethernet link to make sure that I was getting the full performance expected between Deltique and the router.
Had I done a speed test, I likely would have seen a 1MiB/s transfer speed, indicating a link problem, even on Deltique.
-
To rule out a cable problem, I should have tried using a different Gigabit Ethernet cable.
It turns out that the cable was fine, but this is a good troubleshooting step anyway.
Evidence and Explanation
On 30 April 2015, I noticed that my trusty rsync -avzHXShPs
command was maxing out at about 1MiB/s. I ruled out other traffic by monitoring iftop
and by attempting an Internet download on a different computer, which transferred data ten times faster.
I checked the network link with ethtool
:
root@node51 [~]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: No
Speed: 10Mb/s
Duplex: Half
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
drv probe ifdown ifup
Link detected: yes
Aha! 10Mbps at half duplex. That's not normal and is very slow. For reference, this was the expected (good) output, captured by switching Ethernet ports on the router:
root@node51 [~]# ethtool eth0
Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Link partner advertised pause frame use: Symmetric
Link partner advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: pumbg
Wake-on: g
Current message level: 0x00000033 (51)
drv probe ifdown ifup
Link detected: yes
The computer and the router tried to negotiate slower and slower speeds until a reliable one could be established. The slowest one, 10BASE-T half duplex, was picked.
Forcing the full 1000BASE-T full duplex (ethtool -s eth0 speed 1000 duplex full
) expected performance caused the affected computer to drop from the network, strongly suggesting that the router didn't want to talk on a faster link.
Craptop, which is limited to 100Mbps full duplex in the first place, probably didn't have a chance to negotiate 10Mbps half duplex.
Now that I have identified the bad Ethernet port, all I need to do is not use that port or get a new router.
On 27 July 2015, I returned the defective router after installing a new router.
Today, Craptop is running on a fault-tolerant network bond with a 100Mbps full duplex Ethernet connection:
root@node52 [~]# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)
Bonding Mode: fault-tolerance (active-backup)
Primary Slave: eth0 (primary_reselect always)
Currently Active Slave: eth0
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0
Slave Interface: wlan0
MII Status: up
Speed: Unknown
Duplex: Unknown
Link Failure Count: 3
Permanent HW addr: 00:1f:3a:55:dc:0d
Slave queue ID: 0
Slave Interface: eth0
MII Status: up
Speed: 100 Mbps
Duplex: full
Link Failure Count: 3
Permanent HW addr: 00:1e:68:32:4a:b5
Slave queue ID: 0