4 thoughts on - C7, Ipmi, NIC2, Still Fighting

• Once upon a time, mark said:

  No, that does not look right. You have configured the gateway of the IPMI to be the host OS side of the NIC. You can’t do that… in a lot of systems I’ve seen, the IPMI side of the NIC can’t even talk to the host OS on the network.

  The IPMI LAN is an independent controller, separate from the host OS. It does not use any routing/firewall/etc. from the host OS. It is just another device on the network that happens to share the same physical port as the host. It should be configured to talk to the same network gateway and such as the host OS.

  Think of it as if you have two independent systems inside one box; a PC
  and an IPMI device. It is similar to them being two devices with an ethernet switch between them (and another port to the outside world). It doesn’t actually work that way (because they are sharing the physical port), but it is close.

• From previous emails, I gather that mark can’t find the way to set which interface the IPMI BMC uses, so he’s setting the BMC’s IP
  settings to use one of the NICs as a gateway. This is not how you make that setting (it won’t work) but I can see where he’s coming from.

  In my experience, it’s either hard-wired to a particular interface. This should be documented, otherwise you need another computer on the same network or connected with a crossover cable to figure it out.

  Sometimes you can set the interface that IPMI uses in the BIOS or through ‘ipmitool’.

  For Dell hardware, the ipmitool command that ships with CentOS7 has an
  ‘ipmitool delloem lan set ‘ which lets you choose which interface to use.

  =============================================================================# ipmitool delloem lan set

  lan set
  sets the NIC Selection Mode :
  on iDRAC12g OR iDRAC13g :
  dedicated, shared with lom1, shared with lom2,shared with lom3,shared
  with lom4,shared with failover lom1,shared with failover lom2,shared
  with failover lom3,shared with failover lom4,shared with Failover all
  loms, shared with Failover None).
  on other systems :
  dedicated, shared, shared with failover lom2,
  shared with Failover all loms.
  =============================================================================
  If I’m using a system with a shared setup, I don’t set up networking on that interface at all. At least with Dells, even if you set up an IP on the interface, it can’t talk to the BMC from the OS using the shared interface. Ping doesn’t work, ‘ipmitool -I lanplus’ doesn’t work, http doesn’t work. You need to connect from another host. I
  have a private management network that we use for IPMI/iLO systems, because those BMC interfaces are known to be an attack vector.

  Hopefully, this is enough information to explain that you need to find out which interface your IPMI device is using, and to use appropriate IP settings, and to *NOT* use the IP/MAC from any OS interfaces as your IPMI device’s gateway.

• Sorry, the above was a run-on sentence I edited to break up into two paragraphs, but I meant “either it’s hard-wired to a particular interface or you can change it in the BIOS or with ipmitool”.

  Please excuse my confusing email.

• Jonathan Billings wrote:
  
  Thanks for the info; the thing I never understood in the documentation was the business of “shared with lomx” – is that for a second management port?

  Aos, my manager tells me to think of the BMC as a completely separate computer, which it is, but that has *no* contact with the o/s.

  On the other hand… *sigh* – I’ve solved the original issue: y’see, the server’s just below the middle of my chest in the rack, and the fans stick out about a cm or two. I went back into the room last night, with a flashlight… and bent down, and lo and behold, there *was* a perfectly good management port. Connected that, did a warm reboot of the bmc, and all is well.

  My manager, the other admin I work with, and I were all too tall to see the port, so I guess there is some use for short people*….

  mark

  * Of course I’m hearing Randy Newman’s Short People in my head.