Xen Setup Documentation For CentOS?

Home » CentOS-Virt » Xen Setup Documentation For CentOS?
CentOS-Virt 12 Comments

Hi,

what is the proposed way to create domU guests on CentOS 6.5? At first I tried to follow the documentation on the xen project website which recommends using xl. I created a config file and ended up with getting a message that the kernel is not bootable when trying to create a guest. I also had to stop some daemon (xend?) because it said that xl isn`t compatible with it and the daemon must be stopped first.

Then I followed redhat documentation which suggests to use virt-manager
— which doesn`t work because servers don`t have GUIs. So I finally managed to create a guest with virt-install. I can start and stop the guest (which is also running CentOS), though I don`t think this is the right way to create one.

So how exactly are you supposed to create guests?

Now I can`t get the networking to work. I`ve been reading lots of documentation and still don`t understand how that is supposed to work. As far as I understand, you get three different network interfaces:

dom0: a bridge (virbr0)
dom0: a virtual network interface (vifN.X)
domU: a virtual network interface which doesn`t appear to be virtual to
domU

And dom0 keeps it`s own network interface(s), like eth0, which is a physical one.

Is vifN.X the same as eth0 in domU? Or what is it for? If it`s the same, is it supposed to have the same IP on both sides?

How do I make it so that domU has network access (beyond dom0)? How does this network stuff work? Do the virtual devices have to be in different subnets? When they are not, the network becomes reachable via multiple interfaces, and I`m guessing that either packet loops may be created or some paths might be disabled by STP.

Do I have to set up shorewall (or the like) on dom0 to be able to handle network access for guests? Would I need to create a bridge for every guest to be able to handle them separately for firewalling purposes because otherwise packets circumvent firewall rules by directly going over the bridge? If so, why are bridges needed?

I would understand doing things like adding those guests that are visible to the LAN only to the same bridge to have them all reachable likewise. When doing that, it would seem to make sense to use a different subnet for guests in the DMZ.

All the documentation tells you many different things, none of them work and it`s totally confusing. Is there any /good/ documentation somewhere?

12 thoughts on - Xen Setup Documentation For CentOS?

  • Servers *can* have GUI’s. Even if you don’t want to install the full Gnome/KDE/display manager toolkits, it’s possible to set up enough to run X based applications form another host. And virt-manager can be run from a client, with authenticated access to the libvirt server, though I’ve generally not done that.

    If you don’t want to bother with that, you’ll need to learn ‘virsh’, which is the actual tool that libvirt uses to do almost everything.

    [Xen specific network questions skipped, I’ve not been using Xen lately]

    You need to pick. One approach is to set up a bridged connection with one VM, with a second localized VLAN connection, and run shorewall or other firewalls on that VM to manage connections to the rest of the VM’s. This leaves your bandwidth trapped at the capacity of that firewall VM, but it’s not an uncommon soluiton, especially when running complex firewalls and/or proxies in small environments.

    Whether you need bridges then depends on where your firewall is. If it lives on another host on your network, yes, your guests need bridges. If it’s on a VM with two connections, as I described above, it’s potentially much easier to set up on a single firewall VM. But migrating the firewall among multiple VM servers means establishing, and maintaining, a multiple VM server internal network, and if doing that, *THOSE* might mandate bridges.

    It Depends(tm).

    I suggest what you need to accomplish first. Do you have, or want to build, firewalls? Are you isolating DMZ hosts or public facing webservers that need heightened isolation and security?

  • Nico Kadel-Garcia writes:

    Yes, they /can/, and IMO a server shouldn`t have and shouldn`t need one.

    Yes, I tried that and it didn`t work.

    That`s what I`m using now. Is virsh what CentOS users are supposed to use? The documentation on the xen project wiki seems to indicate that xen users are supposed to use xl.

    How do you make such a localised connection?

    In this case, the VM with the firewall needs to have access to an ethernet interface to do pppoe, a second interface to the LAN and to a third one for a DMZ. The VM with the firewall will be on the same physical host as the VMs in the DMZ. There will also be VMs in the LAN
    on the same physical host.

    This seems to require three bridges, and the firewall VM would need access to all of them, unless the dom0 is doing some of the routing. How do you enable access to several bridges for a VM?

    ATM, I`m trying to understand how this networking stuff with bridges basically works.

    Hm, I never thought of that. What kind of limit do I have to imagine?
    For external connections, the highest bandwidth is 1GB LAN. So a limiting factor for internal connections is probably much more important, like a VM accessing a database running in another VM. The database will probably be so small that it can be kept mostly in memory, so disk access won`t slow things down much.

    How much information does the host system have available about the VMs for scheduling purposes? Like when there is a VM for firewalling/routing, a VM with a database and another VM with an application accessing the database, all three VMs need resources. Delay any of them or give them resources at the wrong moment, and in the end performance will be diminished.

    Suppose each of the three VMs has one CPU assigned, and more CPUs are physically available while other VMs happen to be idle. So three CPUs are busy and another five CPUs remain idle so that they are available just in case one of the idle VMs needs to do something? Or will they be used to speed up the running VMs?

    Are there situations in which overcommitment of CPUs is advisable? Like when I have 8 CPUs and 4 VMs, I could assign 2 CPUs to each VM. But when I can expect that 95% or so of the time 2 VMs will be idle when 2
    others are running, won`t I be better off to assign 4 CPUs to each VM?
    Or is this done automatically, and the number of CPUs specified for a VM
    is only a minimum?

    Moving is another thing I haven`t thought about yet. It`s not a requirement in this case, though it would be a good option to have.

    How do you deal with memory overcommitment? Suppose I set up a VM that does the firewalling and routing. Memory requirements for this are low.

    I also want to use squid (2.7), with a fairly large cache and url rewriting (which hasn`t been ported to 3.x yet). That requires memory
    (and file I/O).

    I also need a file server.

    So what do I do? Use one VM for firewall, one for squid, one for a file server that also provides squid with its cache? Or run the file server directly on the host? Or run firewall and squid in the same VM and give this VM some more memory? Or don`t use the file server to supply squid and keep the cache in the same VM squid runs in, perhaps giving it even more memory?

    I could also have the firewall in it`s own VM (which might be a good idea due to security) and use another VM for the routing and to run squid, an MTA and other basic services which are accessible from the LAN
    only. Hm, this actually makes the most sense to me. But then, what do I do with the web server?

    What is good practise?

    Depends on? ;)

    Well, I don`t really want to build firewalls; having one is merely a requirement, and being able to do some traffic shaping can be nice. I`m running a web server which is reachable from the outside (on a non-default port, so it`s not exactly public) and needs to be isolated, and I`m running an MTA. I`m thinking about adding IMAPs so clients can get their mail through that.

    How isolated does a web server need to be? I`m thinking of adding some game servers in their own VM later on, too. But CPU and memory are limited resources …

    Since I need to start somewhere, what needs to be accomplished is probably something like this:

    dom0
    use that as file server directly, or use a VM as file server?

    dom1
    pppoe, firewall

    dom2
    dns, router, squid, MTA, IMAPs, perhaps DHCP

    dom3
    webserver
    game server(s)
    mumble server

    dom4
    internal use (like distcc, a multi-user X server for clients to
    connect to, experimentation …)
    mysql server

    The server has 2×4 CPUs and 8GB of RAM. Unless something speaks against it, I`d prefer to use dom0 as file server because it seems easier to set up, with direct access to the storage volume.

    As to resources:

    |——+——————————+———–+————|
    | dom | RAM | CPUs | scheduling |
    |——+——————————+———–+————|
    | dom0 | how do I assign memory here? | | ++ |
    | dom1 | 512–768 MB | 1 CPU | – |
    | dom2 | 2GB? | 1 CPUs | – |
    | dom3 | 2GB? | 1–3 CPUs | = |
    | dom4 | 4GB? | 3–5 CPUs | = |
    |——+——————————+———–+————|
    | 5 | 9GB | 8 CPUs | == |
    |——+——————————+———–+————|

    The file server would run on dom0. What should I set for dom0_mem?
    1–4GB?

    Dom4 doesn`t need to be up all the time. From what I`ve been reading, you can overcommit memory and it`s a bad idea to overcommit CPUs.

    It it better to give VMs less memory (to some point) and let them use their swap files, or is it better to give them a bit more and overcommit in total (to some point) so that dom0 may swap?

    It also seems that scheduling means that VMs /can/ get more time when they need it and don`t get it when not. How does that interfere with CPU assignments?

    The VMs only need to sustain a very low minimum throughput because usage will be more like short-term spikes on one or another. Would it make sense to overcommit CPUs in this szenario?

    Does this make sense or should I do things differently?

  • TGVlLAoKSSBtYW5hZ2VkIHRvIGdldCB0aGlzIHdvcmtpbmcsIGJ1dCBJJ20gbm90IGluIGZyb250
    IG9mIG15IGNvbXB1dGVyIGF0IHRoZSBtb21lbnQuIMKgSSB1c2VkIEtPQU4gYW5kIENvYmJsZXIg d2l0aCBhIGZldyBjYXZlYXRzLi4uSSdsbCB0cnkgdG8gZm9sbG93IHVwIHRvIHRvbW9ycm93LgoK
    ClNlbnQgZnJvbSBteSBWZXJpem9uIFdpcmVsZXNzIDRHIExURSBTbWFydHBob25lCgotLS0tLS0t LSBPcmlnaW5hbCBtZXNzYWdlIC0tLS0tLS0tCkZyb206IGxlZSA8bGVlQHl1bi55YWdpYmRhaC5k ZT4gCkRhdGU6MDYvMDEvMjAxNCAgODo0NSBQTSAgKEdNVC0wNTowMCkgClRvOiBjZW50b3Mtdmly dEBjZW50b3Mub3JnIApTdWJqZWN0OiBbQ2VudE9TLXZpcnRdIHhlbiBzZXR1cCBkb2N1bWVudGF0
    aW9uIGZvciBjZW50b3M/IAoKSGksCgp3aGF0IGlzIHRoZSBwcm9wb3NlZCB3YXkgdG8gY3JlYXRl IGRvbVUgZ3Vlc3RzIG9uIGNlbnRvcyA2LjU/wqAgQXQgZmlyc3QKSSB0cmllZCB0byBmb2xsb3cg dGhlIGRvY3VtZW50YXRpb24gb24gdGhlIHhlbiBwcm9qZWN0IHdlYnNpdGUgd2hpY2gKcmVjb21t ZW5kcyB1c2luZyB4bC7CoCBJIGNyZWF0ZWQgYSBjb25maWcgZmlsZSBhbmQgZW5kZWQgdXAgd2l0
    aCBnZXR0aW5nCmEgbWVzc2FnZSB0aGF0IHRoZSBrZXJuZWwgaXMgbm90IGJvb3RhYmxlIHdoZW4g dHJ5aW5nIHRvIGNyZWF0ZSBhIGd1ZXN0LgpJIGFsc28gaGFkIHRvIHN0b3Agc29tZSBkYWVtb24g KHhlbmQ/KSBiZWNhdXNlIGl0IHNhaWQgdGhhdCB4bCBpc25gdApjb21wYXRpYmxlIHdpdGggaXQg YW5kIHRoZSBkYWVtb24gbXVzdCBiZSBzdG9wcGVkIGZpcnN0LgoKVGhlbiBJIGZvbGxvd2VkIHJl ZGhhdCBkb2N1bWVudGF0aW9uIHdoaWNoIHN1Z2dlc3RzIHRvIHVzZSB2aXJ0LW1hbmFnZXIKLS0t IHdoaWNoIGRvZXNuYHQgd29yayBiZWNhdXNlIHNlcnZlcnMgZG9uYHQgaGF2ZSBHVUlzLsKgIFNv IEkgZmluYWxseQptYW5hZ2VkIHRvIGNyZWF0ZSBhIGd1ZXN0IHdpdGggdmlydC1pbnN0YWxsLsKg IEkgY2FuIHN0YXJ0IGFuZCBzdG9wIHRoZQpndWVzdCAod2hpY2ggaXMgYWxzbyBydW5uaW5nIGNl bnRvcyksIHRob3VnaCBJIGRvbmB0IHRoaW5rIHRoaXMgaXMgdGhlCnJpZ2h0IHdheSB0byBjcmVh dGUgb25lLgoKU28gaG93IGV4YWN0bHkgYXJlIHlvdSBzdXBwb3NlZCB0byBjcmVhdGUgZ3Vlc3Rz PwoKCk5vdyBJIGNhbmB0IGdldCB0aGUgbmV0d29ya2luZyB0byB3b3JrLsKgIElgdmUgYmVlbiBy ZWFkaW5nIGxvdHMgb2YKZG9jdW1lbnRhdGlvbiBhbmQgc3RpbGwgZG9uYHQgdW5kZXJzdGFuZCBo b3cgdGhhdCBpcyBzdXBwb3NlZCB0byB3b3JrLgpBcyBmYXIgYXMgSSB1bmRlcnN0YW5kLCB5b3Ug Z2V0IHRocmVlIGRpZmZlcmVudCBuZXR3b3JrIGludGVyZmFjZXM6CgoKZG9tMDogYSBicmlkZ2Ug KHZpcmJyMCkKZG9tMDogYSB2aXJ0dWFsIG5ldHdvcmsgaW50ZXJmYWNlICh2aWZOLlgpCmRvbVU6
    IGEgdmlydHVhbCBuZXR3b3JrIGludGVyZmFjZSB3aGljaCBkb2VzbmB0IGFwcGVhciB0byBiZSB2
    aXJ0dWFsIHRvCsKgwqDCoMKgwqAgZG9tVQoKCkFuZCBkb20wIGtlZXBzIGl0YHMgb3duIG5ldHdv cmsgaW50ZXJmYWNlKHMpLCBsaWtlIGV0aDAsIHdoaWNoIGlzIGEKcGh5c2ljYWwgb25lLgoKSXMg dmlmTi5YIHRoZSBzYW1lIGFzIGV0aDAgaW4gZG9tVT/CoCBPciB3aGF0IGlzIGl0IGZvcj/CoCBJ
    ZiBpdGBzIHRoZQpzYW1lLCBpcyBpdCBzdXBwb3NlZCB0byBoYXZlIHRoZSBzYW1lIElQIG9uIGJv dGggc2lkZXM/CgpIb3cgZG8gSSBtYWtlIGl0IHNvIHRoYXQgZG9tVSBoYXMgbmV0d29yayBhY2Nl c3MgKGJleW9uZCBkb20wKT/CoCBIb3cKZG9lcyB0aGlzIG5ldHdvcmsgc3R1ZmYgd29yaz/CoCBE
    byB0aGUgdmlydHVhbCBkZXZpY2VzIGhhdmUgdG8gYmUgaW4KZGlmZmVyZW50IHN1Ym5ldHM/wqAg V2hlbiB0aGV5IGFyZSBub3QsIHRoZSBuZXR3b3JrIGJlY29tZXMgcmVhY2hhYmxlIHZpYQptdWx0
    aXBsZSBpbnRlcmZhY2VzLCBhbmQgSWBtIGd1ZXNzaW5nIHRoYXQgZWl0aGVyIHBhY2tldCBsb29w cyBtYXkgYmUKY3JlYXRlZCBvciBzb21lIHBhdGhzIG1pZ2h0IGJlIGRpc2FibGVkIGJ5IFNUUC4K
    CkRvIEkgaGF2ZSB0byBzZXQgdXAgc2hvcmV3YWxsIChvciB0aGUgbGlrZSkgb24gZG9tMCB0byBi ZSBhYmxlIHRvIGhhbmRsZQpuZXR3b3JrIGFjY2VzcyBmb3IgZ3Vlc3RzP8KgIFdvdWxkIEkgbmVl ZCB0byBjcmVhdGUgYSBicmlkZ2UgZm9yIGV2ZXJ5Cmd1ZXN0IHRvIGJlIGFibGUgdG8gaGFuZGxl IHRoZW0gc2VwYXJhdGVseSBmb3IgZmlyZXdhbGxpbmcgcHVycG9zZXMKYmVjYXVzZSBvdGhlcndp c2UgcGFja2V0cyBjaXJjdW12ZW50IGZpcmV3YWxsIHJ1bGVzIGJ5IGRpcmVjdGx5IGdvaW5nCm92
    ZXIgdGhlIGJyaWRnZT/CoCBJZiBzbywgd2h5IGFyZSBicmlkZ2VzIG5lZWRlZD8KCkkgd291bGQg dW5kZXJzdGFuZCBkb2luZyB0aGluZ3MgbGlrZSBhZGRpbmcgdGhvc2UgZ3Vlc3RzIHRoYXQgYXJl CnZpc2libGUgdG8gdGhlIExBTiBvbmx5IHRvIHRoZSBzYW1lIGJyaWRnZSB0byBoYXZlIHRoZW0g YWxsIHJlYWNoYWJsZQpsaWtld2lzZS7CoCBXaGVuIGRvaW5nIHRoYXQsIGl0IHdvdWxkIHNlZW0g dG8gbWFrZSBzZW5zZSB0byB1c2UgYQpkaWZmZXJlbnQgc3VibmV0IGZvciBndWVzdHMgaW4gdGhl IERNWi4KCkFsbCB0aGUgZG9jdW1lbnRhdGlvbiB0ZWxscyB5b3UgbWFueSBkaWZmZXJlbnQgdGhp bmdzLCBub25lIG9mIHRoZW0gd29yawphbmQgaXRgcyB0b3RhbGx5IGNvbmZ1c2luZy7CoCBJcyB0
    aGVyZSBhbnkgL2dvb2QvIGRvY3VtZW50YXRpb24Kc29tZXdoZXJlPwoKCi0tIApLbm93bGVkZ2Ug aXMgdm9sYXRpbGUgYW5kIGZsdWlkLsKgIFNvZnR3YXJlIGlzIHBvd2VyLgpfX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpDZW50T1MtdmlydCBtYWlsaW5nIGxp c3QKQ2VudE9TLXZpcnRAY2VudG9zLm9yZwpodHRwOi8vbGlzdHMuY2VudG9zLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2NlbnRvcy12aXJ0Cg=

  • I understand how frustrating it can be to be dealing with old /
    inaccurate documentation. But I’m not sure how we’re supposed to help you if you don’t give any details about what you did and exactly how it failed. If the instructions you followed don’t work, then either:
    * You misunderstood something / accidentally left out a step /
    mistyped something from the documentation.
    * There’s a bug in the CentOS implementation of Xen that needs to be fixed
    * There’s a bug in the documentation that needs to be fixed

    If you describe which bit of documentation on the Xen website you tried to follow, what you were trying to do, and what happened, then we can figure out which of those it is and address the issue.

    The xend thing is an unfortunate — as a project we’re trying to move away from it, but there are still a large number of CentOS users who use it. Trying to make both new users and old users happy is a bit of a hard balancing act.

    -George

  • George Dunlap writes:

    I was merely trying to create a VM on a CentOS host, using xen. Hence my question what “the CentOS way” of doing this (without a GUI) is.

    By trial and error, I found that creating them with virsh-install works. Yet it seems to me as if virsh is an additional layer which makes dealing with VMs a lot easier at the cost of increased resource usage. My intention was to avoid this and to create VMs “the xen way” — which apparently doesn`t work with CentOS.

    There seem to be at least three different ways to create VMs, two of which have been used at different times with xen, while virsh tries to give users a common interface which isn`t exactly needed when you want to stick to xen and therefore do things “the xen way”.

    But then, I guess every distribution has their own way to make things more complicated, so a common interface like virsh seems to be the most efficient way.

    Well, I didn`t really care about the documentation that didn`t work. Like I said, using xl didn`t work because it only says the kernel can`t be booted; xm seems to be deprecated (yet still used with CentOS), and virsh-install works.

    There could be different packages in CentOS, one set featuring xend, the other featuring whatever replaces it.

    I`ve come to like virsh because it`s pretty straightforward and actually works. From that point of view, it doesn`t matter to me whether xend is used or xl or xm or something else. It only matters when I`m concerned about what might be more efficient or have other advantages relevant to me.

    Since xend and xm are appartenly deprecated and will or have been replaced, I don`t see much point in learning them and figuring out how they might be advantageous. I rather learn to use what is current and may be used in the future.

  • The Xen project itself doesn’t have anything to build disk images;
    just as KVM itself doesn’t have anything to build disk images. It leaves that to higher-level tools.

    I know some people use virsh to install the guest, and then use the xl command-line to manage it after that; but I haven’t tested that.

    It would be good to have recommendations on the wiki for a simple, standardized way to create guests in CentOS.

    *I* care about the documentation that didn’t work, so that other people don’t trip over the same thing. :-) If you’ve walked this path and become frustrated, there are probably a dozen other people who have also walked it and just not said anything.

    Thanks,
    -George

  • George Dunlap writes:

    Creating a VM and getting the network to work are the biggest problems I`m having with this. Even with virt-install, you cannot use a CentOS
    minimal CD ISO because it`ll say that something is missing. With the full installer DVD ISO you get to the point where the installer asks you where the installation media is — and then you`re screwed because you
    *must* have network access from within the VM at that point.

    The installer lets you set up network access, but only once. If you don`t get it right the first time, you have to start over.

    Nobody tells you how to get the network access to work. Suppose the dom0 has an IP of 192.168.10.10 and you tell the installer to use
    192.168.10.20. That seems to make sense, but there`s no way to get network access from within the VM with that. I still don`t understand why and why this is so awfully difficult.

    You have to use something like 192.168.20.20 for the VM and make sure routing on dom0 is set up accordingly. You have to use an http server or nfs on dom0 to make the installation media available available to the VM. If the http/nfs server is not on dmo0, you have to set up forwarding to make it reachable for the VM. You have to make sure to disable the rudimentary firewall dom0 has to be able to access it from within the VM.

    You have to figure out that the installer doesn`t want the installation media, which is the ISO file. No, you have to mount the ISO, copy everything in it to a directory and then export that directory via nfs so that finally, if you got network access to work, the installer can proceed.

    Then you need to figure out how to get the bridge set up when starting the system. I created an ifcfg file for it and am told that bringing up that interface is delayed because it doesn`t seem to exist.

    There doesn`t seem to be any documentation about the file apparently used by xend to configure the VM. Documentation tells you to use virsh edit , and with that, I`m seeing my changes *not* being applied even after restarting the VM. I`d edit the file directly
    (somewhere under /etc/xend/domains or so), but without documentation about what`s in it, I rather avoid that.

    How do you set a memory range (min–max) for the VM with virsh? There doesn`t seem to be a way to do that, and even if there was, how do I
    make it so that changes through virsh edit are applied?

    It seemed to me that xend and xl are incompatible.

    Yes, that`s exactly what I`m asking about all the time: The CentOS way of creating guests.

    As it is, you have to piece together hundreds of bits of documentation and by trial and error somehow figure out some way that gets you what you want.

    I`d gladly tell you which documentation it was if I could remember. Some of the documentation on the xen wiki seems to be outdated or a mess because it evolved over time and how things are supposed to be done has changed.

    Almost all documentation recommends you to create a file to keep the VM
    in. I was planning to do that until I found some other documentation telling me that it`s a bad idea because file access like that is relatively slow and you`re much better off using LVM volumes instead. Besides being faster, they have other advantages as well, so I`m doing it that way.

    Once I figured out how to have VMs started automatically, I need to find out how to give a VM access to a physical interface to do pppoe over it and how to give it access to two more virtual interfaces …

    Is there a good documentation that explains this network stuff with VMs?
    I still don`t really have a clue how to do that.

  • George Dunlap writes:

    To give an example:

    https://www.CentOS.org/docs/5/html/5.1/Deployment_Guide/s1-networkscripts-static-routes.html

    [root@charon ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
    DEVICE=”eth0″
    BOOTPROTO=none DNS1=”192.168.178.20″
    IPADDR=”192.168.1.1″
    NETMASK=”255.255.255.255″
    NM_CONTROLLED=no ONBOOT=”yes”
    TYPE=”Ethernet”
    UUID=”1b645d25-9f66-4335-ba0b-939cdd9f553f”

    [root@charon ~]# cat /etc/sysconfig/network-scripts/route-eth0
    192.168.178.0/24 via 192.168.1.1 eth0
    default 192.168.178.200 dev eth0

    [root@charon ~]# service network restart Shutting down interface eth0: [ OK ]
    Shutting down loopback interface: [ OK ]
    Bringing up loopback interface: [ OK ]
    Bringing up interface eth0: Determining if ip address 192.168.1.1 is already in use for device eth0… Error: either “to” is duplicate, or “eth0” is a garbage. Error: either “to” is duplicate, or “192.168.178.200” is a garbage.
    [ OK ]
    [root@charon ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface
    [root@charon ~]#

    https://docs.fedoraproject.org/en-US/Fedora/13/html/Deployment_Guide/s1-networkscripts-static-routes.html

    “If the default gateway is already assigned from DHCP, the IP command arguments format can cause one of two errors during start-up, or when bringing up an interface from the down state using the ifup command:
    “RTNETLINK answers: File exists” or ‘Error: either “to” is a duplicate, or “X.X.X.X” is a garbage.’, where X.X.X.X is the gateway, or a different IP address. These errors can also occur if you have another route to another network using the default gateway. Both of these errors are safe to ignore.”

    This error is not “save to ignore” because it makes the network unreachable. Without the duplication and only the default route specified, I`m getting the same error, though only once, and still no routing.

    Then from https://docs.fedoraproject.org/en-US/Fedora/13/html/Deployment_Guide/s1-networkscripts-static-routes.html:

    “You can also use the network/netmask directives format for route-interface files. The following is a template for the network/netmask format, with instructions following afterwards:
    […]”

    Let`s see …

    [root@charon ~]# cat /etc/sysconfig/network-scripts/route-eth0
    ADDRESS02.168.178.0
    NETMASK0%5.255.255.0
    GATEWAY02.168.1.1

    [root@charon ~]# service network restart Shutting down interface eth0: [ OK ]
    Shutting down loopback interface: [ OK ]
    Bringing up loopback interface: [ OK ]
    Bringing up interface eth0: Determining if ip address 192.168.1.1 is already in use for device eth0…
    [ OK ]
    [root@charon ~]# route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use Iface
    192.168.178.0 192.168.1.1 255.255.255.0 UG 0 0 0 eth0
    [root@charon ~]#

    So why does the second approach work and the first one doesn`t? It looks still strange since eth0 is configured with 192.168.1.1 anyway –

  • syntax error. you need a “dev” before eth0. which actually is useless to append anyway because the interface is already specified in the file name

    syntax error. you miss a “via” before the IP of the gateway

    explained by the syntax error from the first line in route-eth0

    explained by the syntax error from the second line in route-eth0

    explained by the fact that your route-eth0 file doesn’t contain any correct line

  • Manuel Wolfshant writes:

    You’re right, I overlooked the ‘dev’.

    Why doesn’t the error message simply say “syntax error” and perhaps even points out that “dev” might be missing? There is no ‘to’ in the configuration file anywhere, and saying that “eth0” might be “a garbage”
    isn’t helpful in any way. This isn’t any better than failing silently or just printing “error”.

    I would like to make a bug report about this so that the useless error message may be changed. But what package should the report refer to?

  • Because the program “ip” is pretty cryptic, despite being way more powerful than most people know

    Each line of the route-eth* file(s) is passed ad-literam to ip route commands so all error messages that you see come from ip. Examine
    /etc/sysconfig/network-scripts/ifup-routes for details. In your case, you should test the content of route-eth* by using: ip route add
    $EachLineOneByOne

    You could file a RFE either against iproute which actually triggers the messages you’ve seen or against initscripts (ifup-routes is provided by it – you could ask for a better parser ).

  • Manuel Wolfshant writes:

    The program is too powerful and cryptic to print reasonable error messages …

    IIRC, I’ve been reading that route shall be replaced with ip, and I
    don’t like that idea. Route is sufficiently confusing and works fine.

    I checked if there is a ‘to’ in the file when I got that message, and there was none.

    Thanks :) Improving ip so that it can print useful error messages seems to make more sense than involving a special parser for the particular purpose of these initscripts.