VLAN Issue

Home » CentOS » VLAN Issue
CentOS 22 Comments

Hello all,

I have a machine running CentOS 6.6 connected to a port on a Cisco Catalyst
3750 series switch. That port is part of VLAN 48. I have VLAN 48 on the CentOS machine too.

The IP network on VLAN 48 is 192.168.48.0/255.255.255.0. The address on the CentOS side is 192.168.48.101, the address on the Linux end ought to be
192.168.48.100.

When I only bring up eth0.48 VLAN device with the IP2.168.48.100 I have no connectivity. If I bring it up along with eth0 with another VLAN 48
address assigned to it (for instance, 192.168.48.99) I do have connectivity.

Also, strangely enough, sometimes to get things going I have to disconnect the Linux host from the switch – physically detach the wire and reconnect it again.

Be that as it may when I just bring up the VLAN by itself I have thus far been unable to get anywhere.

Has anybody seen a situation like this? Does anybody have an explaination for it?

Thanks.

Boris.

22 thoughts on - VLAN Issue

  • I think you always have to bring up the underlying eth device to activate a related eth.nn vlan. The base device would not normally have an IPADDR, though, unless it is for an untagged vlan 0. Assuming the connected switch port is configured as a trunk, you shouldn’t see vlan 48 addresses on the base (untagged) device.

  • Less,

    You are 100% right. Of course I brought up my eth0 – but, like you said, with no IP. Meanwhile, I brought up eth0.48 with 192.168.48.100.

    However, until I would bring up eth0 with an IP address (any in the network) I would have no connection. Why? That’s what I fail to understand.

    Boris.

  • Doesn’t make sense to me – I think I’ve done it both ways
    (with/without a vlan 0 address). I didn’t think it took anything special except the VLAN=yes in the file and the .number in the DEVICE(and file) name.

  • Are you actually seeing VLAN tagged traffic, or is the cisco switch just providing a normal stream?

    At work we have hundreds of VLANs, but the servers don’t get configured for this; we just configure them as normal; ie eth0. The network infrastructure does the VLAN decoding, the server doesn’t have to.

    Try configuring the machine as if it was a real LAN and forget about the VLAN.

    If that doesn’t work then what does ‘tcpdump -i eth0’ show you?

  • Steve,

    Thanks, makes sense.

    I just don’t see why I have to effectively waste an extra IP address to get my connection established.

    Boris.

  • We have lots of servers with a similar setup (i.e. tagged vlans and no ip on eth0) and this works just fine.

    What is the actual vlan configuration on your switchport?

    Regards,
    Dennis

  • Hi Boris, what I’d like to know is the actual VLAN configuration of the switch port (link-type and tagged and untagged VLANs). When I look at the switchport coniguration here I get (among other things):

    … Port link-type: trunk
    Tagged VLAN ID : 8, 1624
    Untagged VLAN ID : 10

    Here is my suspicion:
    Your ports have an access link-type with an untagged VLAN ID of 48. That would explain why the moment you configure an IP from that VLAN on eth0
    you get connectivity because then the packets the Linux box sends are untagged as the switch would expect them to be. If you only put an address on eth0.48 then the packets get tagged by Linux but if the switch port is not configured to receive the packets for VLAN 48 as tagged then it will simply drop these packets and you will not get connectivity.

    So getting the actual VLAN config of the switch port would help to determine if the switch actually expects to receive the packets the way you send them from the Linux box.

    Regards,
    Dennis

    So if you

  • Andrew and Dennis are spot on. Their conclusions about your server being connected to an access port and not a trunk port would be my conclusion as well.

    Additionally, the switch should gripe about 802.1q BPDUs. Check the in-memory system log (or syslog server if you have configured that).

    show logging | i 1Q

    Example:
    1w1d: %SPANTREE-2-RECV_1Q_NON_TRUNK:
    Received 802.1Q BPDU on non trunk FastEthernet0/2 on vlan 100.

    +1
    Let’s see the config for the switch port your server is connected to.

  • If you are on a trunk port then your machine needs to be configured for VLANs. If you are not on a trunk port then your machine needs to be configured normally.

    It _sounds_ like you have configured your machine for VLANs but are not on a trunk port.

  • Because a trunk port will “trunk” the vlan.

    A VLAN is basically a 4 byte “tag” that gets injected into the packet header when the packet enters the VLAN network. When we trunk a VLAN we say to the switch “pass packets on VLAN x but do not strip the tag out”.

    You can either terminate the VLAN at the switch port (untagged) which will strip out the VLAN tag or you can pass the packet containing the VLAN tag to the computer or other device(tagged/trunk). This device can then pull out the tag. On linux this mechanism is done by an 8021q VLAN interface.

    Hope this is useful.

    ta

    Andrew

  • Just to add to that – normally if a host only needs to be on one subnet you would use an access port on the switch to select a single vlan and deliver those packets untagged so the host does not need to care about tags or vlan numbers. And to that end, switches default to treating everything as access ports on native/untagged vlan 0
    unless configured otherwise. However, if the host needs interfaces on multiple subnets, you can do it on a single network connection by giving it a trunk connection from the switch and letting it split out the vlan interfaces internally.

  • Thank you everyone.

    OK, the mystery deepens, I guess. The machine does need to support several VLAN’s, it is currently on a trunkport (8021q encapsulated), it made it into the ARP table – which I specifically tested for by physically unplugging the table, clearing the ARP table and plugging it back in.

    The ARP table currently looks like this:

    hq#show arp Protocol Address Age (min) Hardware Addr Type Interface Internet 192.168.48.100 0 0025.6440.0301 ARPA Vlan48
    Internet 192.168.48.101 – 001b.906a.bcc4 ARPA Vlan48
    Internet 192.168.48.1 0 0025.6440.063f ARPA Vlan48
    Internet 192.168.2.52 0 0025.6440.0547 ARPA Vlan2
    Internet 192.168.3.1 – 001b.906a.bcc2 ARPA Vlan3
    Internet 192.168.2.1 – 001b.906a.bcc1 ARPA Vlan2
    Internet 192.168.7.1 – 001b.906a.bcc3 ARPA Vlan7
    hq#

    The network config on the machine currently looks like this: it has nothing assigned to eth0, eth0.48 = 192.168.48.100/24, eth0.49 = 192.168.49.100/24, eth0.50 = 192.168.50.100/24.

    And – even though the ARP table seems to be OK – there is no connectivity!

    Boris.

  • And additionally here are the detailed port configs on the switch end:

    hq>show interface Gi1/0/3 switchport Name: Gi1/0/3
    Switchport: Enabled Administrative Mode: trunk Operational Mode: trunk Administrative Trunking Encapsulation: dot1q Operational Trunking Encapsulation: dot1q Negotiation of Trunking:

  • OK, thanks again for all your help.

    I have resolved this, finally. The problem was that I configured VLAN 48 as the native VLAN on the trunk port.That was a mistake as apparently the native VLAN is the one where Cisco does not bother to tag packets.

    For now I set the native VLAN to VLAN 1 and that works.

    Cheers,

    Boris.

  • +1

    If it were me, I’d opt for setting the native vlan to 48 for that port. It’s simpler and avoids having vlan1 to deal with.

    Boris could just set what vlans are allowed on the trunk port to his server. Just allow vlans 48, 49, and 50 and not others

    ! by default your switch trunks on vlan 1 to 4094
    ! now to allow it only on the three vlans you specifically specified
    (48,49,50)
    switchport trunk allowed vlan remove 1-47,51-4094
    ! if you chose to tell it not to trunk any vlans, you’d disconnect your telnet/ssh
    ! session as well as cause a service outage … so don’t do that!
    !
    ! also realize that Cisco smuggles some data via VLAN1 [0], so there still will likely be traffic on VLAN1
    !
    ! now that port should not be trunking on ALL vlans … just 48,49,50
    show int Gi1/0/3 switchport show int Gi1/0/3 trunk

    As far as security goes … Leaving vlan1 usable when it does not need to be is akin to locking most of the doors at your home, but not all of them.

    1) by default (most?) switches have all ports in vlan1 … so somebody plugs in a new switch and could potentially communicate with your server.
    2) If someone compromises that server, now they have a trunk port to have lots of fun with (create more vlan interfaces and sniff/spoof traffic).

    [0]
    https://supportforums.cisco.com/discussion/9118321/disabling-vlan1-across-trunks

  • I think the common uses of VLANs are to segregate traffic to reduce collisions, and to segment networks for security. If you’ve added VLAN
    1 as the native VLAN, you might be exposing this host to attacks from that VLAN (assuming you gave it an address), or you might be exposing other hosts on the VLAN to attacks from that host if it’s compromised. If your security policy is based on the policy of least privilege, then it’s best not to provide a host with VLANs that it doesn’t use/need.

LEAVE A COMMENT