12 thoughts on - Need Help Getting Two NICs To Work On CentOS 7

• We’re missing a lot of details, but to speak generally, I’d look at the bridging on the host if you’re confident that the interfaces are configured properly inside the host.

  Also, have you looked at iptables?

• The last time I did that I did it on CentOS 5. Here are my notes from back then.

  First, add two lines to /etc/iproute2/rt_tables, so it looks like this:

  #
  # reserved values
  #
  255 local
  254 main
  253 default
  0 unspec
  #
  # local
  #
  #1 inr.ruhep
  50 access1
  60 access2

  Then add the following routes:

  ip route add 1.2.3.0/24 dev eth1 table access1
  ip route add default via 1.2.3.1 dev eth1 table access1
  ip rule add from 1.2.3.178/32 lookup access1

  ip route add 1.2.3.0/24 dev eth2 table access2
  ip route add default via 1.2.3.1 dev eth2 table access2
  ip rule add from 1.2.3.180/32 lookup access2

  Of course, in this example 1.2.3.178 is the IP address of the first nic and 1.2.3.180 is the address of the second nic.

• –tLnMwI0sFh9gN4HDJ1XXt3m1ponx49Dul Content-Type: multipart/alternative;
  boundary=”————46EE1C80D773706FC7971B24″

  This is a multi-part message in MIME format.
  ————–46EE1C80D773706FC7971B24
  Content-Type: text/plain; charset=windows-1252
  Content-Transfer-Encoding: quoted-printable

  I’ve done this recently on C7. The key words for your Google Fu is policy based routing. That ensures the incoming packets come back out the same interface they came in on. Mind you, depending on the routers in the way this may not be a problem. CISCO ASAs get really picky but otherwise there may be other problems to deal with as well.

  If you’re using Network Manager then the package:
  NetworkManager-config-routing-rules will help.

  After that you create the rules and routes files in
  /etc/sysconfig/network-scripts

  So in my case I have:

  * enp3s0: 192.168.20.253 (the default gateway on this subnet is
  192.168.20.2)
  * vlan22@enp3s0: 192.168.22.253 (the default gateway on this subnet is
  192.168.22.1)

  [root@host network-scripts]# cat route-enp3s0
  192.168.20.0/24 dev enp3s0 tab 20
  default via 192.168.20.2 dev enp3s0 tab 20

  [root@host network-scripts]# cat rule-enp3s0
  from 192.168.20.0/24 tab 20 priority 20

  [root@host network-scripts]# cat route-vlan22
  192.168.22.0/24 dev vlan22 tab 22
  default via 192.168.22.1 dev vlan22 tab 22

  [root@host network-scripts]# cat rule-vlan22
  from 192.168.22.0/24 tab 22 priority 22

  Wireshark can be really helpful here as it can confirm that the
  “unpingable” NIC is actually receiving the packets that you think it is. And hopefully in that case can also confirm if the echo packets are going out the alternate NIC. If there are no incoming packets then the PBR is no real help at all and you’d need to look further upstream.

  Hope that helps,
  -pete

  ————–46EE1C80D773706FC7971B24

• You need one rule file per interface, which directs traffic out the appropriate interface based on the source address of the packet:

  https://blogs.oracle.com/networking/entry/advance_routing_for_multi_homed

• policy based routing is useful concept but in my mind should be the very last resort. It is the kernel’s networking functions which are responsible for routing tables construction and if you can shed more details (like mentioned earlier) on what can(not)
  ping what(VM hosting box <-> VM guest / outside internal net
  <-> VM / etc.) centbuddies could share more concrete thoughts then. Basically – with policy/source based routing you need to be very careful, test everything – best is to leave it to the kernel and only use main table, kernel should be the smartest, for more complex setups maybe think “routing daemons”.

• Hello there,

  What is the hypervisor that hosts the VM? What does ifconfig show on it?

  Boris.

• Hello all

  Thank you for the replies. I ran into some issues with the policy based routing which I will explain in a few. First off to answer each one of your questions:

  Digimer: No I didnt take a look at IPTables

  Peter Brady: Thank you for the example, I tried that but it failed and I
  will explain in a minute.

  Frank Cox : That works but how do I make it persistent across reboots?

  Boris Epstein: I am using Hyper-V and its getting annoying lol

  Ok so here is how I have set everything up:

  my /etc/iproute2/rt_tables:

  #
  # reserved values
  #
  255 local
  254 main
  253 default
  0 unspec
  #
  # local
  #
  #1 inr.ruhep
  300 NIC1
  310 NIC2

  my /etc/sysconfig/network-scripts/route-eth0 file:

  168.87.147.0/24 dev eth0 table 300
  default via 168.87.147.1 dev eth0 table 300

  /etc/sysconfig/network-scripts/route-eth1 file:

  10.20.50.0/24 dev eth1 table 310
  default via 10.20.50.1 dev eth1 table 310

  My /etc/sysconfig/network-scripts/rule-eth0:

  from 168.87.147.33/32 table 300
  to 168.87.147.33 table 300

  My /etc/sysconfig/network-scripts/rule-eth1:

  from 10.20.50.90/32 table 310
  to 10.20.50.90 table 310

  now after implementing this and restarting NetworkManager, when I run ‘ip rule list’, I get the following:

  0: from all lookup local
  32764: from all to 10.20.50.90 lookup NIC2
  32765: from 10.20.50.90 lookup NIC2
  32766: from all lookup main
  32767: from all lookup default

  and when i run ‘ip route’ , i get the following:

  default via 168.87.147.1 dev eth0 proto static metric 1024
  10.20.50.0/24 dev eth1 proto kernel scope link src 10.20.50.90
  168.87.147.0/24 dev eth0 proto kernel scope link src 168.87.147.33

  yet when I try to ping from another server to 10.20.50.90 it will not ping at all. Also, whenever I reboot the VM, eth1 switches over to DHCP and I
  lose my IP configuration. After I reset the IP Configuration back to Manual and reenter the IP, CentOS creates a new interface file called ifcfg-Wired_Connection-1 and places the IP configuration for the interface into that file. Both virtual NICs are set to Static Mac Addresses, so I
  dont know why it keeps creating another interface file. Any ideas on what I am doing wrong here? All help is greatly appreciated!

  Thanks!

• Nothing obvious. Since your interfaces have static configurations, I’d suggest turning off NetworkManager and turning on the “network” service to determine whether or not that works correctly after a reboot.

  Assuming it does, fixing the issue might be as simple as getting NetworkManager to not rename one interface to “wired connection 1”.

• Hello Gordon

  Thank you for your response. I disabled NetworkManager and started the network service, then I rebooted the VM and now ‘ip rule show’ has the rules for both interfaces showing but I have no network connectivity even when I restart the network service. Should I reenable NetworkManager now?
  How would I go about getting NetworkManager to stop creating new Interface files and naming them Wired_connection-*?

  Thanks!

• Yeah, the switch is just a test to see if the problem is specific to NetworkManager. It seems that you have other problems as well.

  Before you do that, post the output of the following:

  ip route show

  ip route show table 300

  ip route show table 301

  ip rule show

  cat /etc/sysconfig/network-scripts/ifcfg-eth0

  cat /etc/sysconfig/network-scripts/route-eth0

  cat /etc/sysconfig/network-scripts/rule-eth0

  cat /etc/sysconfig/network-scripts/ifcfg-eth1

  cat /etc/sysconfig/network-scripts/route-eth1

  cat /etc/sysconfig/network-scripts/rule-eth1

• Hello

  I think i figured it out. I had forgotten to install NetworkManager-config-routing-rules and then enable and start the NetworkManager-dispatcher service. I tried that on a different test machine and I am able to ping both of the IPs from another machine just fine. Only a few things I have noticed: No matter how many times I
  restart the NetworkManager-dispatcher service, if I reboot the system, and run a systemctl status NetworkManager-dispatcher.service, the service shows as Inactive (Dead) even though it has been enabled. Also whenever I
  reboot the VM, after logging into the GUI, I get a pop up that says failed to activate network connection , yet I still have network connectivity. Any ideas on what I can do to fix these things? Also overall, because I am using a static IP config for both NICs, is it better to just disable NetworkManager and use network scripts instead?

  Thanks!!

• That’s normal. It’s a dbus-invoked service. You don’t need to manually enable it. It runs a command when it’s invoked, and then terminates.
  There isn’t a daemon process associated with the service.

  Not really. The log might tell you why it thinks an interface wasn’t successfully enabled.

  Not generally. If you find the old network service works better for your use case, it’s available. I use it on a small number of hosts where NM bugs interfere with normal operation, but I’m waiting for the
  7.3 updates (where NM has been rebased to a newer version) to see if I
  can stop doing that.