GigE -> 100Mb Problems

Home » CentOS » GigE -> 100Mb Problems
CentOS 12 Comments

I’ve got a pair of identical CentOS 6.7 servers, with SuperMicro X8DTE-F
motherboards, these have 2 each Intel 82574L ethernet ports. The eth0
ports are plugged in with 10′ runs of brand new cat 5e cable to a Cisco Nexxus 9000 switch (provided by the data center).

These servers keep coming up at 100baseT rather than gigE. I’ve swapped ports and cables with a different server, that server gets gigE but these supermicro’s are stuck at 100baseT regardless of the port or cableused.

‘ethtool eth0’ says its gigE, but ‘mii-tool -v eth0’ says its 100baseT
and doesn’t even list gigE as an option, and the cisco switches see it as 100baseT too.

anyone else run into anything like this before?

12 thoughts on - GigE -> 100Mb Problems

  • oh.

    # mii-tool -v eth0
    eth0: negotiated, link ok
    product info: vendor 00:50:43, model 11 rev 1
    basic mode: autonegotiation enabled
    basic status: autonegotiation complete, link ok
    capabilities: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
    advertising: 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
    flow-control

    # ethtool eth0
    Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Supported pause frame use: No
    Supports auto-negotiation: Yes
    Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: Yes
    Speed: 1000Mb/s
    Duplex: Full
    Port: Twisted Pair
    PHYAD: 1
    Transceiver: internal
    Auto-negotiation: on
    MDI-X: on (auto)
    Supports Wake-on: pumbg
    Wake-on: g
    Current message level: 0x00000007 (7)
    drv probe link
    Link detected: yes

  • Yeah, the entire “net-tools” package is deprecated. I tend to forget which of the two (ethtool or mii-tool) is in that set.

    # Avoid using any of these:
    $ rpm -ql net-tools

  • /bin/dnsdomainname
    /bin/domainname
    /bin/hostname
    /bin/netstat
    /bin/nisdomainname
    /bin/ypdomainname
    /sbin/ether-wake
    /sbin/ifconfig
    /sbin/ipmaddr
    /sbin/iptunnel
    /sbin/mii-diag
    /sbin/mii-tool
    /sbin/nameif
    /sbin/plipconfig
    /sbin/route
    /sbin/slattach

    ok, so the mii-* stuff is deprecated (as is route, ifconfig, netstat, and arp? sigh).

    apparently the network administrator went ahead and forced the switch port to use gigE, so its no longer in the ‘broken’ state of autonegotiating 100baseT.

  • Just for comparison this is a server that is set to auto negotiate

    [root@sch-mwg-01 access.log]# mii-tool eth0
    eth0: negotiated 100baseTx-FD, link ok
    [root@sch-mwg-01 access.log]# ethtool eth0
    Settings for eth0:
    Supported ports: [ TP ]
    Supported link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: Yes Advertised link modes: 10baseT/Half 10baseT/Full
    100baseT/Half 100baseT/Full
    1000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: Yes Speed: 1000Mb/s Duplex: Full Port: Twisted Pair PHYAD: 1
    Transceiver: internal Auto-negotiation: on MDI-X: off Supports Wake-on: pumbg Wake-on: g Current message level: 0x00000007 (7)
    drv probe link Link detected: yes

    Note that mii-tool is reporting incorrectly. That server is currently processing ~500Mbit/s. Of the tools listed above, the only one I still extensively use is netstat as ss, its replacement, is IMO not very nice.

    Tris

    *************************************************************
    This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify postmaster@bgfl.org

    The views expressed within this email are those of the individual, and not necessarily those of the organisation
    *************************************************************

  • mii-tool doesn’t support gigE. Maybe, they’re not negotiating:
    “ethtool -s eth0 autoneg on”, you can renegotiate with “ethtool -r eth0”


    Marcelo

    “¿No será acaso que esta vida moderna está teniendo más de moderna que de vida?” (Mafalda)

  • Hi,

    the cisco switch or server NICs.

    Please test that if both the server are communicating with each other at
    1Gbps or not via “iperf” tool.

    If above gives result of 1Gbps then it will eliminate the NICs problem then you know that it is a problem with cisco switch only.

    –Regards Ashishkumar S. Yadav

  • after they forced the cisco ports to gigE, I was seeing 200-400Mbps in iPerf, which was odd. servers were both very lightly loaded.

    BUT… the switch ports kept going offline on us. Note I have no admin access to the switch, its managed by IT so I have to go through channels to get anything. I asked what error codes were causing the ports to go offline but haven’t heard back. as of right now, both servers are offline, (I can reach their IPMI management controller, and remotely log onto the console just fine, but the ports show no link).
    When I was in the DC yesterday, I switched ports, same problem, I also switched the network cable with a different (HP) server, it had no problems on the same cable+port thats giving these supermicro servers problems.

    I’d chalk it up to a bad NIC, but two identical servers with two nic’s each all have this problem, so its got to be something else, some weirdness with the 82574L as implemented on these SuperMicro X8DTE-F
    servers running CentOS 6.7 ?!? In our old DC, these servers ran rock solid for several years without any network issues at all, in that rack I had a Netgear JGS524

  • Not finding anything exactly like this… Closest I could find is CSCuu81949

    Open a Cisco TAC case and upload a Nexus 9000 tech support (`tac-pac`) to investigate further.

    Is “port security” enabled on these ports?
    Does this port double as a LOM/IPMI port?
    What driver is being used `ethtool -i eth#`?
    Any OS bonding?
    Any switch port-channel/vPC?

    That NIC chipset is a few years old so it’s not like that NIC/OS/switch is a combo that hasn’t been tried/tested.

  • I don’t know, asking LAN operations

    no, boards have dedicated IPMI port, which doesn’t go offline

    e1000m

    no

    I don’t believe so.

    yeah, about as bog stock normal a config as I’ve seen.