Are Xen And CentOS Incompatible?

Home » CentOS-Virt » Are Xen And CentOS Incompatible?
CentOS-Virt 12 Comments

Hi,

I’m trying to pass a physical network interface through to a domU.

This seems to be impossible because the way xen wants to do it is incompatible with the way CentOS wants to do it.

I followed documentation on http://www.xen-support.com/?p1 and tried booting with ‘pciback.permissive pciback.hide=(06:00.0)’. This gives a hint in dmesg “kernel: xen-pciback: backend is vpci”, and the device is still visible in dom0. So this obviously doesn’t work.

Following http://wiki.xen.org/wiki/Xen_PCI_Passthrough:

[root@heimdall ~]# xl pci-assignable-add 06:00.0
xend is running, which may cause unpredictable results when using this xl command. Please shut down xend before continuing.

(This check can be overridden with the -f option.)
[root@heimdall ~]#

There doesn’t seem to be any documentation about what xend does or is for and how to pass a physical network interface through with it.

I’m starting to think that using CentOS for a server os is an extremely bad choice because nothing works. It’s not like a Linux distribution but like a mess of pieces from several unrelated puzzles thrown together randomly.

12 thoughts on - Are Xen And CentOS Incompatible?

  • Huh?

    The device should be visible in the dom0 – even when it is for passthrough.

    But irrespective of that – the steps mentioned there are out of date. The correct option should be ‘xen-pciback.hide=(06:00.0) xen-pciback.permissive=1’

    which is mentioned in that link.

    Did you just try using the same parameter but replace ‘xl’ with ‘xm’?

  • Konrad Rzeszutek Wilk writes:

    Why should it be visible when it’s hidden?

    That’s one of the problems: Xen is very poorly documented.

    I replaced CentOS with debian and finally got it to work. Things are much easier with debian.

    I don’t remember … It seemed I had to use virsh since nothing else worked on CentOS, and virsh doesn’t let you seem to get networking stuff to work.

    CentOS and xen just don’t go along with each other. The server is all on debian now.

  • I don’t remember the post at which I stopped paying attention to you. It seems you just don’t follow instruction well, or can’t think outside the box, or can’t communicate accordingly.

    Good luck with that.

  • The ‘hide’ part is to hide it from the device drivers in the initial domain – dom0. That is so that they will not try to use it – as we plan to pass them to a guest. We need it in the dom0 to do admin type work – FLR it, etc.

    Any help in improving the documentations would be appreciated. Every month we run ‘Documentation days’ and any work – either on Wiki, manuals, docs, etc would be quite appreciated.

    Please see http://wiki.xenproject.org/wiki/Xen_Project_Document_Days

    Glad to hear you solved the problems.

  • Konrad Rzeszutek Wilk writes:

    With Debian, it’s not visible in dom0 when the passthrough works. That’s how I found out that it does work to begin with, and it makes sense to me.

    What does FLR mean? And how do you do something with a device for which no drivers are loaded? I’d find it rather unusual to have a device without drivers and even be able to use it; such devices usually don’t show up.

    If I have some time, I might make a writeup about how to set up what I
    did. But it seems I’m using an outdated version of xen, which is what comes with Debian, so by the time I’d finish the writeup, it would be outdated and contribute to confusion more than do any good.

    And considering xen, I don’t really know anything. I figured out that passthrough doesn’t work out of the box on Debian because the module for the device was loaded from the initrd.img before the xen-pciback module and made a bug report because you’re supposed to be able to use files in
    /etc/modprobe.d which can specify dependencies and when you do that, you can’t have that just overridden or there’s no point in doing that –

  • That is a surprise. If you do ‘lspci’ in your dom0 do you see the device (06:00.0)?

    Function Level Reset.

    You pass the device to a guest so it can load the drivers and the initial domain (dom0) won’t use it.

  • Konrad Rzeszutek Wilk writes:

    Lspci still shows it, and the device (eth1) is invisible.

    Hm, xen kinda makes the cpus and their power management invisible, too:

    root@heimdall:~# xenpm get-cpufreq-para
    [CPU0] failed to get cpufreq parameter
    […]
    root@heimdall:~# xenpm get-cpufreq-states root@heimdall:~#

    So I guess it could as well make it so that lspci doesn’t show passed-out devices.

    BTW, getting some info in dmesg might be nice, like a message saying
    “xen-pciback: device 06:00.0 can be passed through to guests”. We could actually see right away if it did work or not. That a device disappears isn’t too great as indication, especially not when lspci still lists it.

    Of course, you could use the command (which I don’t remember) to show devices that can be passed through. But that may just work as well as
    ‘xenpm get-cpufreq-states’: Apparently, there aren’t any CPUs …

  • I am wondering if you are using an older kernel. The xen-acpi-processor driver should be loaded which would give the C and P states to the hypervisor. Which in turn would result in those above commands providing the right data.

    You just need to boot with ‘debug’ – and it should tell you that a device is being assigned to another guest (when assigning). Also at bootup it will tell you that it is seizinging.

    Just do ‘dmesg | grep pciback’ and you will get it.

    Such as xl or xm pci-list-assignable?

    See if xen-acpi-processor is loaded or built in.

  • Konrad Rzeszutek Wilk writes:

    Linux heimdall 3.2.0-4-amd64 #1 SMP Debian 3.2.57-3+deb7u2 x86_64 GNU/Linux

    This is what comes in Debian. Unfortunately, this kernel crashes when I’m copying data to a domU NFS server over the network :(( I need to find out how to get some useful information out of it to make a bug report.

    How do I know whether the xen-acpi-processor driver is loaded or not?

    Ok, I enabled debugging. Maybe that also helps to get some more info about the crashes.

    yes

    Unless it’s called “processor”, it doesn’t seem to exist:

    root@heimdall:~# lsmod |grep proc processor 28149 1 acpi_cpufreq thermal_sys 18040 1 processor root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64 |less
    root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name “*proces*”
    /lib/modules/3.2.0-4-amd64/kernel/drivers/acpi/processor.ko root@heimdall:~# find /lib/modules/3.2.0-4-amd64/ -name “*xen*”
    /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netfront.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback
    /lib/modules/3.2.0-4-amd64/kernel/drivers/net/xen-netback/xen-netback.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen
    /lib/modules/3.2.0-4-amd64/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/watchdog/xen_wdt.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntalloc.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-gntdev.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-evtchn.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xen-pciback/xen-pciback.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs
    /lib/modules/3.2.0-4-amd64/kernel/drivers/xen/xenfs/xenfs.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkfront.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback
    /lib/modules/3.2.0-4-amd64/kernel/drivers/block/xen-blkback/xen-blkback.ko
    /lib/modules/3.2.0-4-amd64/kernel/drivers/pci/xen-pcifront.ko root@heimdall:~# grep -i proces /boot/config-3.2.0-4-amd64
    CONFIG_BSD_PROCESS_ACCT=y CONFIG_BSD_PROCESS_ACCT_V3=y
    # Processor type and features CONFIG_ACPI_PROCESSOR=m CONFIG_ACPI_PROCESSOR_AGGREGATOR=m
    # Audio decoders, processors and mixers root@heimdall:~# grep -i xen /boot/config-3.2.0-4-amd64
    CONFIG_XEN=y CONFIG_XEN_DOM0=y CONFIG_XEN_PRIVILEGED_GUEST=y CONFIG_XEN_PVHVM=y CONFIG_XEN_MAX_DOMAIN_MEMORY8
    CONFIG_XEN_SAVE_RESTORE=y
    # CONFIG_XEN_DEBUG_FS is not set CONFIG_MICROCODE_XEN=y CONFIG_PCI_XEN=y CONFIG_XEN_PCIDEV_FRONTEND=m CONFIG_XEN_BLKDEV_FRONTEND=m CONFIG_XEN_BLKDEV_BACKEND=m CONFIG_NETXEN_NIC=m CONFIG_XEN_NETDEV_FRONTEND=m CONFIG_XEN_NETDEV_BACKEND=m CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y CONFIG_HVC_XEN=y CONFIG_XEN_WDT=m CONFIG_XEN_FBDEV_FRONTEND=y
    # Xen driver support CONFIG_XEN_BALLOON=y
    # CONFIG_XEN_BALLOON_MEMORY_HOTPLUG is not set CONFIG_XEN_SCRUB_PAGES=y CONFIG_XEN_DEV_EVTCHN=m CONFIG_XEN_BACKEND=y CONFIG_XENFS=m CONFIG_XEN_COMPAT_XENFS=y CONFIG_XEN_SYS_HYPERVISOR=y CONFIG_XEN_XENBUS_FRONTEND=y CONFIG_XEN_GNTDEV=m CONFIG_XEN_GRANT_DEV_ALLOC=m CONFIG_SWIOTLB_XEN=y CONFIG_XEN_PCIDEV_BACKEND=m root@heimdall:~#

  • lsmod

    But it looks like v3.4 and later were the kernels that started having this driver. That would explain why it does not exist as you are using
    3.2.

  • Konrad Rzeszutek Wilk writes:

    Well, yes, I didn’t know what the name of this module exactly is. There’s also the “processor” module.

    Because of the crashes, I’ve upgraded to a 3.14.5-1~bpo70+1 kernel from Debian backports. That one does have the “xen-acpi-processor” module, and it’s working. I don’t know yet if it’ll crash, though.