Finally Switching From Xen To KVM – Question About Networking

Home » CentOS-Virt » Finally Switching From Xen To KVM – Question About Networking
CentOS-Virt 27 Comments

I had so much trouble putting CentOS 6 guest VMs on a CentOS 5 host that I finally switched to a CentOS 6 host.

I’ve not needed more that test VMs, so I’ve used Virtual Machine Manager on the old system, which worked pretty well, so I decided to create my first KVM guest machine. I noticed when I created it, I only had the options of NAT for my network interface, so I used that (obvious).

Well, after starting the VM, I find I don’t have connectivity with that interface. Reading, I find examples where I need to create bridges perhaps. Xen did most of this for me, so it’s a little new to me.

Can anyone throw me a clue, please?

steve campbell

27 thoughts on - Finally Switching From Xen To KVM – Question About Networking

  • The host has a device named virbr0 that is installed during system installation. It also has a network device vnet0. There are no files in
    /etc/sysconfig/network-scripts for these.

    Shouldn’t I be able to use the virbr0 virtual bridge for this? I’ve tried setting up the VM’s device with all of the options that is listed, but to no avail.

    Should I need to set up another bridge for this?

    And thanks for the link.

    steve

  • virbr0 is created and managed by libvirtd. If you open Virtual Machine Manager, connect to localhost and then double-click on ‘localhost’, you will see a tab for creating/managing bridges (NAT’ed, generally). I
    disable ‘virbr0’ as NAT’ing is generally not what I want.

    The ‘vnetX’ devices are dynamically created to link a VM’s interface to a bridge. Think of them as virtual network cables. They get created and destroyed as needed.

  • Sorry, but I’m confused:

    My host server has a real NIC and IP address with a real gateway to the outside:
    virtbr0 IP: 192.168.122.1
    Host IP: 10.0.5.16
    Gateway IP: 10.0.5.1
    on eth0 and this works

    My VM server has all fake stuff currently:
    Host IP: 10.0.5.17
    Gateway IP: 10.0.5.1
    on eth0 and this is like NIC without a cable.

    So I need to create a bridge device on both the host and VM (lets say I
    name it br1). I change the eth0 config file on both host and VM to point to br1 and give the br1 config file on both host and VM the correct IP. But won’t this just let the two talk to each other. How will the VM
    server get outside?

    steve

  • Do you have only one public IP address and do you want to share that between your host and your VM(s)? In this case you must use NAT config.

    If you have multiple public IPs and you want to assign one of those to your host and another of those to your guest (VM) then you must create a bridge such as br0 (or br1, it doesn’t matter how you number it).

  • The bridge is created only on the host. Think of the bridge as being like a virtual switch. When a VM is created, you tell it to connect to the bridge, similar to how you would plug a physical wire into a real switch. That provides the link to the network, and then you configure the virtual server’s network just as if it was a real network.

    On the host, you don’t set the IP on the ethX device, instead you tell ethX to connect to the bridge (look at ‘bond2’ in the link above for the
    ‘BRIDGE=”…”‘ line). Then you move the host’s IP address/config to the bridge itself (look at the ‘vbr2’ in the link above).

    If you’re still stuck, tell me the IP addresses you want to set in the host and VM and I’ll see if I can bang out a couple sample ‘ifcfg-X’ files.

  • Steve Campbell wrote:

    Steve, I just put up my notes for how to set up an EL6
    host as well as how to create a guest. You’ll note that it is terse and it expects you to already know what options you want. However if you follow it step by step by copying and pasting the appropriate lines, you’ll have your system set up in no time.

    http://wiki.CentOS.org/HowTos/KVM

    Good luck! And let me know if you have any problems with any of the commands in the wiki.

    c

  • Steve Campbell writes:

    This bridge is like a stick you poke through a wall into another room
    (domU –> dom0). At the same time, it *is* a bridge, i. e. it works like a hub in that it *does* connect *all* network(s) with each other via the devices that are added to the bridge. That is regardless of what IP addresses these devices have or what networks they “belong” to.

    To make it more confusing, the bridge itself is a network device.

    Suppose you have the physical eth0 on the machine, with 192.168.1.1. Add eth0 to the bridge and you don’t have network access anymore (which is really confusing).

    Now do not give eth0 an IP address but give the bridge 192.168.1.1 and, with eth0 attached to it, you suddenly have network access again (which is also confusing). Both the VM and the host are reachable from the outside that way.

    The other end of the bridge shows up as a network interface (eth0) in the VM. You can give it an IP address like 192.168.100.1, and that address is reachable (from the outside of the VM) via the bridge in the host. You may have to add a route in the host for it, though, and for the VM, the host is the gateway.

    For your purposes, you can probably ignore the virtual interfaces. The duplication with bridges and interfaces still doesn’t make sense to me
    … What’s the difference between a dangling bridge and a virtual interface?

  • I’m running into some problems with my config files where I get a message that says the network can’t determine my interface names. I’ll look further, but the IPs above are what I need. 10.0.5.1 is a firewall that NATs the 10.0.5.0/24 network to another outside public firewall.

    Thanks steve

  • Stuck in a rut now. Restarting the network indicates :

    Device eth0 does not seem to be present, delaying initialization.

    I thought it might be due to the fact I didn’t have the “TYPE” in the ifcfg-eth0 file, but after adding that, I still get the error.

    I’m using the same hardware address that was in the original eth0 file. My bridge device starts fine, but I’ll tailor it to your specs.

    Thanks steve

  • NM_CONTROLLED=no might help with that.

    That is fine.

    Also, you might want to check the contents of your
    /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address for eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0 file?

  • I turned off and stopped NetworkManager and had already specified this. They are the same except the udev file has lower case letters in it. I appreciate your time on this.

    steve

  • All is well on the host now, and I’ve brought up the VM and configured it to use the bridge. I can ping the host from the VM but I have no access beyond the local network. The host sees the world, but the VM
    only sees the 10.0.5.0/24 network.

    steve

  • Did you set the gateway in your VM to 10.0.5.1 ?
    Can you ping the gateway ( ping 10.0.5.1 ) from the VM?
    Do you have a firewall configured in your VM (or host) ? If so, try to disable the firewall temporarily for testing.

  • The gateway is set. Can’t ping the gateway. Firewalls (iptables and ip6tables) are stopped and off. Previous post suggested the route. I’ll check it.

    steve

  • Add the following lines to your /etc/sysctl.conf file on your host:

    net.ipv4.ip_forward = 1
    net.ipv4.conf.default.rp_filter = 0
    net.ipv6.conf.all.disable_ipv6 = 1
    net.ipv6.conf.default.disable_ipv6 = 1

    then issue the following command:

    sysctl -p

  • Still no joy. I’m installing another VM just to see if having the bridge defined before installation would make a difference. I find it hard to imagine all of this can’t be done will creating the VM (Xen seemed to do it all for you).

    So now, I’ve got the route, gateway, bridge, and eth0/1 defined on the original. I can ping both the host and the VM from each other. The host can see the world through the gateway. The VM can’t see the gateway or the world.

    steve

  • What you did on the host machine, only has to be done once when you setup the system to be used as a KVM host server.

    How did you come up with the IP address ( 10.0.5.17 ) for the VM ? Are you sure it is not blocked somehow by your gateway ( at 10.0.5.1 ) ?

    Also, maybe your gateway ( at 10.0.5.1 ) is doing MAC address filtering?
    Since your VM has its own MAC address, this could be a problem.

  • The gateway is a firewall. It’s set up to route and NAT from 10.0.5.0/24
    to another inside network on 10.0.0.0/24. This is the true inside network that is NATted to the public with our public IPs. It accepts all traffic from the 10.0.5.0 network.

    steve

  • I created another VM, this time the bridge was available when it was created, an it’s working. I’ll delete the first VM and re-create it and see if that doesn’t resolve it.

    Thanks for all the help and tutoring on this.

    steve