Eth1 Problem On CentOS-6.3

Home » CentOS » Eth1 Problem On CentOS-6.3
CentOS 14 Comments

I am trying to transport a dd image between to hosts over a cross linked gigabit connection. Both hosts have an eth1 configured to a non routable ip addr on a shared network. No other devices exist on this link.

When transferring via sftp I received a stall warning. Checking the logs I see this:

dmesg | grep eth

e1000e 0000:00:19.0: eth0: (PCI Express:2.5GT/s:Width x1)
00:1c:c0:f2:1f:bb e1000e 0000:00:19.0: eth0: Intel(R) PRO/1000 Network Connection e1000e 0000:00:19.0: eth0: MAC: 7, PHY: 8, PBA No: FFFFFF-0FF
r8169 0000:01:00.0: eth1: RTL8168d/8111d at 0xffffc9000187c000,
00:0a:cd:1d:44:fe, XID 081000c0 IRQ 31
r8169 0000:01:00.0: eth1: jumbo features [frames: 9200 bytes, tx checksumming: ko]
ADDRCONF(NETDEV_UP): eth0: link is not ready device eth0 entered promiscuous mode r8169 0000:01:00.0: eth1: invalid firwmare r8169 0000:01:00.0: eth1: unable to load firmware patch rtl_nic/rtl8168d-1.fw (-22)
r8169 0000:01:00.0: eth1: link down r8169 0000:01:00.0: eth1: link down ADDRCONF(NETDEV_UP): eth1: link is not ready r8169 0000:01:00.0: eth1: link up ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready br0: port 1(eth0) entering learning state eth1: no IPv6 routers present eth0: no IPv6 routers present br0: port 1(eth0) entering forwarding state r8169 0000:01:00.0: eth1: link down r8169 0000:01:00.0: eth1: link up r8169 0000:01:00.0: eth1: link down r8169 0000:01:00.0: eth1: link up

This may, or may not, be related to this bug:

https://bugzilla.kernel.org/show_bug.cgi?id411

Is there some way to confirm whether or not this is the case. Is there a work-around for it if it is this bug? If it is not then has anyone any idea what is happening and how to fix it?

14 thoughts on - Eth1 Problem On CentOS-6.3

  • Elrepo.org has updated drivers for both e1000e and r8169 devices (I’m guessing it’s probably the kmod-r8168 you’d need). You could try these to see if they resolve the issue.

    If you want more help identifying the correct driver for your hardware, see FAQ #4 here:

    http://elrepo.org/tiki/FAQ

    Hope that helps.

  • The network card for eth1 seems to have disappeared somehow. On the problem host:

    # /sbin/lspci -nn | grep -i net
    00:19.0 Ethernet controller [0200]: Intel Corporation 82567V-2 Gigabit Network Connection [8086:10ce]
    #

    On another but nearly identically configured host (same MB and additional NIC:

    # /sbin/lspci -nn | grep -i net00:19.0 Ethernet controller [0200]:
    Intel Corporation 82567V-2 Gigabit Network Connection [8086:10ce]
    01:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller [10ec:8168] (rev
    03)
    04:04.0 Serial controller [0700]: NetMos Technology PCI 9835 Multi-I/O
    Controller [9710:9835] (rev 01)

    Where did eth1 on the first host go? How do I get it back? A restart?

  • Faulty hardware maybe? Try a reboot and see if it reappears. If it’s located on a card try reseating the card (although I suspect this is an integrated NIC on the motherboard?).

    The chipset is not necessarily the same in the second example (different revision); RTL8111/8168B is not RTL8168d/8111d. They probably do use the same driver but I’d need to see the Vendor:Device ID pairing to know for sure.

  • James,

    James B. Byrne wrote:

    I have a bad feeling about this, to quote Han Solo. lspci not showing it?
    First thing I’d do is power cycle the box. If it shows up, fine. If not, or if it shows up, then vanishes in the near future… is this an on-the-m/b NIC? If so, start spec’ing out a new m/b or box.

    mark

  • Eth1 is an xpci card sold by StarTech. A system with an identical card reports this:

    for BUSID in $(/sbin/lspci | awk ‘{ IGNORECASE=1 } /net/ { print $1
    }’); do /sbin/lspci -s $BUSID -m; /sbin/lspci -s $BUSID -n; done

    00:19.0 “Ethernet controller” “Intel Corporation” “82567V-2 Gigabit Network Connection” “Intel Corporation” “Device 0028”
    00:19.0 0200: 8086:10ce

    01:00.0 “Ethernet controller” “Realtek Semiconductor Co., Ltd.”
    “RTL8111/8168B PCI Express Gigabit Ethernet controller” -r03 “Realtek Semiconductor Co., Ltd.” “TEG-ECTX Gigabit PCI-E Adapter [Trendnet]”
    01:00.0 0200: 10ec:8168 (rev 03)

    04:04.0 “Serial controller” “NetMos Technology” “PCI 9835 Multi-I/O
    Controller” -r01 -p02 “LSI Logic / Symbios Logic” “2S (16C550 UART)”
    04:04.0 0700: 9710:9835 (rev 01)

    The 4 port serial controller on the second host does not have an equivalent card installed on the host that no longer recognizes eth1. The integrated NI is eth0 for both hosts and this i/f is integrated on the Intel motherboard. The motherboards are the same model in both hosts. Both system are configured as KVM hosts. Both are running CentOS-6.3

  • OK, I’d definitely try reseating the card and if you still get no joy I’d swap it out for a replacement.

    kmod-r8168 is the correct driver for this device. If you’d like to try the updated driver from elrepo, just install kmod-r8168 and reboot.

    lsmod should then show r8168 in use and not the native kernel driver r8169. Many have found this driver to be more reliable than the native kernel driver, assuming the hardware isn’t defective.

    Hope that helps.

  • I swapped the suspect card and rebooted the host. After some fussing about with udev I managed to get the new card recognized as eth1 (vice eth2 as udev kept insisting).

    I will do a transfer test later today and see if it stays up. The original failed in the midst of an sftp transfer.

  • James B. Byrne wrote:

    Yup. I assume you edited /etc/udev/rules.d/70-persistent-net.rules, got rid of the lines for the old card, and told it the new one was eth1?

    Good luck.

    mark

  • Having replaced the suspect card and rebooted the host I see these messages in /var/log/messages repeated over and over:

    Aug 15 07:17:10 vhost01 ntpd[2044]: Listening on interface #62 eth1, fe80::20a:cdff:fe1d:32e7#123 Enabled Aug 15 07:17:10 vhost01 ntpd[2044]: Listening on interface #63 eth1,
    192.168.216.41#123 Enabled Aug 15 07:19:41 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:19:41 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:19:41 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:19:43 vhost01 ntpd[2044]: Deleting interface #62 eth1, fe80::20a:cdff:fe1d:32e7#123, interface stats: received=0, sent=0, dropped=0, active_time3 secs Aug 15 07:19:43 vhost01 ntpd[2044]: Deleting interface #63 eth1,
    192.168.216.41#123, interface stats: received=0, sent=0, dropped=0, active_time3 secs Aug 15 07:20:13 vhost01 kernel: r8169 0000:01:00.0: eth1:
    RTL8168d/8111d at 0xffffc9001258c000, 00:0a:cd:1d:32:e7, XID 081000c0
    IRQ 30
    Aug 15 07:20:13 vhost01 kernel: r8169 0000:01:00.0: eth1: jumbo features [frames: 9200 bytes, tx checksumming: ko]
    Aug 15 07:20:14 vhost01 kernel: r8169 0000:01:00.0: eth1: invalid firwmare Aug 15 07:20:14 vhost01 kernel: r8169 0000:01:00.0: eth1: unable to load firmware patch rtl_nic/rtl8168d-1.fw (-22)
    Aug 15 07:20:14 vhost01 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:20:14 vhost01 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:20:14 vhost01 kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready Aug 15 07:20:16 vhost01 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:20:16 vhost01 kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready Aug 15 07:20:20 vhost01 ntpd[2044]: Listening on interface #64 eth1, fe80::20a:cdff:fe1d:32e7#123 Enabled Aug 15 07:20:20 vhost01 ntpd[2044]: Listening on interface #65 eth1,
    192.168.216.41#123 Enabled Aug 15 07:28:14 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:28:14 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:28:14 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:28:16 vhost01 ntpd[2044]: Deleting interface #64 eth1, fe80::20a:cdff:fe1d:32e7#123, interface stats: received=0, sent=0, dropped=0, active_timeG6 secs Aug 15 07:28:16 vhost01 ntpd[2044]: Deleting interface #65 eth1,
    192.168.216.41#123, interface stats: received=0, sent=0, dropped=0, active_timeG6 secs Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1:
    RTL8168d/8111d at 0xffffc90012588000, 00:0a:cd:1d:32:e7, XID 081000c0
    IRQ 30
    Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1: jumbo features [frames: 9200 bytes, tx checksumming: ko]
    Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1: invalid firwmare Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1: unable to load firmware patch rtl_nic/rtl8168d-1.fw (-22)
    Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:29:27 vhost01 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:29:27 vhost01 kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready Aug 15 07:29:30 vhost01 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:29:30 vhost01 kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready Aug 15 07:29:33 vhost01 ntpd[2044]: Listening on interface #66 eth1, fe80::20a:cdff:fe1d:32e7#123 Enabled Aug 15 07:29:33 vhost01 ntpd[2044]: Listening on interface #67 eth1,
    192.168.216.41#123 Enabled Aug 15 07:31:54 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:31:54 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:31:54 vhost01 lldpad[1836]: evb_ifdown:port eth1 remove failed Aug 15 07:31:55 vhost01 ntpd[2044]: Deleting interface #66 eth1, fe80::20a:cdff:fe1d:32e7#123, interface stats: received=0, sent=0, dropped=0, active_time2 secs Aug 15 07:31:55 vhost01 ntpd[2044]: Deleting interface #67 eth1,
    192.168.216.41#123, interface stats: received=0, sent=0, dropped=0, active_time2 secs

    So, what is going on? This host is the first one of two nearly identically configured machines. The second host does not report or evidence any problem with its eth1.

  • James B. Byrne wrote:

    My eyes uncrossed, and I saw, buried in there, the firstlink, above, and the last. You might want to see if a) the 8168d firmware patch will work on that card; b) vhost – it’s a virtual host? perhaps it’s trying to load the firmware patch to the real NIC, and as a guest VM, it doesn’t have rights?

    mark

  • r8169 0000:01:00.0: eth1: invalid firwmare

    I cannot seem to find a fix for this even given the references provided. However, I have discovered that eth1 has problems on the second host as well, it just does not generate the same messages:

    [root@vhost02 ~]# grep eth /var/log/messages Aug 13 17:10:18 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 13 17:16:19 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 11:23:39 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 11:23:41 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 11:37:42 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 11:37:44 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 11:38:51 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 11:38:53 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 11:40:17 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 11:40:19 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 11:40:50 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 11:40:52 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 14 15:26:20 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 14 15:26:22 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 05:28:39 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 05:28:42 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 05:40:32 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 05:40:34 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 05:47:34 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 05:47:36 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 05:48:21 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 05:48:23 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:16:53 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:16:54 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:18:35 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:18:37 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:22:27 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:22:29 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:31:36 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:31:38 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:40:27 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:40:29 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:41:40 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:41:42 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 06:49:39 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 06:49:41 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:01:59 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:02:01 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:04:08 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:04:10 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:07:28 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:07:30 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:09:54 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:09:56 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:12:07 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:12:09 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:13:58 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:14:00 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:17:04 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:17:06 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:20:14 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:20:16 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 07:29:28 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 07:29:30 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 09:31:56 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 09:31:58 vhost02 kernel: r8169 0000:01:00.0: eth1: link up Aug 15 09:33:18 vhost02 kernel: r8169 0000:01:00.0: eth1: link down Aug 15 09:33:20 vhost02 kernel: r8169 0000:01:00.0: eth1: link up

    So, my questions are: What is the problem and how do I fix it for both hosts?

  • Have you tried the updated driver I suggested?

    Set up elrepo.org repository and then do:

    yum install kmod-r8168

    reboot and test.

    If it doesn’t work you can uninstall it and you’ve lost nothing but 10
    minutes of your time.

  • vhost is a kvm host system, not a virtualized guest. As I wrote elsewhere, I installed the kmod package from elrepo that handles this device, which solution I believe you may have originally suggested to me. In any case, that seems to have solved the problem (fingers crossed).

    Thanks for the help. I appreciate it very much.

    Sorry about your eyes. But when things are totally unfamiliar to me I
    hesitate to trim error logs lest out of ignorance I remove what is really important.

LEAVE A COMMENT