45 thoughts on - A Question About 7

• Check the progress on http://seven.CentOS.org/

  As for the latter question – too early to tell.

• Not yet publicly available.

  I’ve heard they have something running in the development environment, but that they’re still working on getting some of the RPMs to build. That’s a prerequisite for generating ISOs.

  Anything that makes a RHEL7 -> C7 transition more difficult also makes the reverse transition more difficult. In this new world of Red Hat /
  CentOS collaboration, that would be a Bad Thing. I’d be surprised if the transition was more difficult than just a forced upgrade of the CentOS-release RPM.

  How bad is the worst case — reinstall the OS and rebuild the software
  — anyway? By doing your initial work on the RHEL 7 beta, you learn what you need to know to quickly redo the work on CentOS 7.

  Post an actual error message, and someone may be able to help.

• Is there anything to simplify the process of duplicating the set of installed packages when you didn’t pay that much attention the first time around? It seems like taking the list from ‘rpm -qa’ on a running machine and feeding it to ‘yum install ….’ on a new minimal install should get pretty close, but then you need to find all of your locally modified config files. Most of that should be under
  /etc/sysconfig for an easy diff, but not everything.

• Warren Young wrote:
  
  install Evergreen, F/OSS library software, on a system, and it’s a nightmare. whateverthelatestanimalis. The PostgreSQL 9, and nothing else. PGSQL 9 was not a big deal to install, but the other stuff…. Even trying to build it in /usr/local is a royal mess:though I’ve got the dbi installed, ./configure can’t find it.

  That’s not a problem, well, the issue, as I said, is that configure can’t find the interface, even when I’ve given it the correct paths. Can’t really just give it to you, since a) I’m at work, in the DC ‘burbs, and b)
  the clubhouse is up in Baltimore. Not building it at home – I’ve got other things to build (like xtrackcad)….

  Thanks for the offer, though. I’m sure there’s either some environment variable it isn’t happy with, or is missing, or an option isn’t right to tell it what path to use.

  mark

• Les Mikesell wrote:
  — anyway? By doing your initial work on the RHEL 7 beta, you learn what you need to know to quickly redo the work on CentOS 7. installed packages when you didn’t pay that much attention the first time around? It seems like taking the list from ‘rpm -qa’ on a running machine and feeding it to ‘yum install ….’ on a new minimal install should get pretty close, but then you need to find all of your locally modified config files. Most of that should be under

  Interesting note: I just started looking at the docs for 7. and two things: one, of course, preupgrade is in, and second, something that, if I
  read it correctly, called zstream, which will give you a report of what preupgrade would do, and list things that would have to be done my hand, so you can judge the difficulty of upgrading in place.

  mark

• I suspect it’s not actually that simple. I think you’d need to do a fair bit of processing on the rpm -qa list to be able to build a yum command that will succeed. Consider the RPM “provides” mechanism, which allows two different RPMs to provide the same capability under different names. {redhat,CentOS}-release is this way, for example.

  One of the reasons I’m playing with RHEL 7 right now is that my end purpose is to be able to modify the documentation and scripts our system installers will use to build new CentOS 7 systems. So, I’m already recording all of the changes needed, partly on paper, partly in a Subversion repository. My RHEL 7 VM is disposable.

  Whenever I’m faced with a system with unknown changes which has to be nuked and rebuilt, I tar up /etc, /home, and *maybe* /var and/or
  /usr/local.

  I usually don’t bother with /var, since the irreplaceable things under
  /var get backed up separately: DB tables, the web tree, etc.

  There are exceptions. The Bind zone files on the primary DNS server are essentially unique, for example. (The cached version on the secondary DNS server(s) isn’t identical to the primary copy. It’s stripped of comments, the formatting is a bit different, etc.)

  I scp the backup tarball off to a file server somewhere, then replace the hard drive and start fresh. The extra HDD and disk space for the backups are cheap insurance.

  The replaced HDD can be given another mission once you’re satisfied that everything’s migrated. Put it in an external USB case and use it for the new system’s off-site backup, for example.

  Not a lot of things. I regularly modify things under

  /etc/ssh/
  /etc/httpd/
  /etc/pki/
  /etc/{init.d, rc.d}/ (via chkconfig and yum)
  /etc/yum.repos.d/
  /etc/samba/

  Plus there’s plenty at the top level that changes occasionally:

  /etc/{hosts,services}
  /etc/{group,passwd,shadow}
  /etc/sudoers

  No, I’ll stand by my current practice: tar up all of /etc and /home, at minimum.

• Were you aware that RHEL 7 now uses “Consistent Network Device Naming”
  (http://goo.gl/Z0ydDF) in more situations? It was optional in RHEL 6
  (http://goo.gl/TiuTP9) but is all but enforced in RHEL 7.

  Everyone, drop a tear for the dead “eth0”. We will miss you, eth0!

• Well one bit is /root/anaconda-ks.cfg to see what the kickstart to duplicate the install of the system would look like …

  As for config files well behaved RPMs should mark their config files in the manifest so a rpm -Va and parsing the output to look for config files changed from install should help …

  In addition you could do something like find /etc -type f -exec rpm -qf {}
  \; | grep -i ‘not owned by’ to find any config files in /etc that are properly marked in the spec files for a package…

  There are probably a few special cases like PostgreSQL having the postgres,conf file in the data directory for the service but you should be able to catch most config files to back up by combining the two tricks…

• Haven’t played much with it in CentOS. In Fedora, at present, it is a bit of pain as both biosdevname and systemd have something to do with it, making it less consistent than ever.

  So, to remove it, one has to both run rpm -e biosdevname and add to
  /etc/deafaults/grub kernel line net.ifnames=0.

• Fpr 7:
  ” If the system’s BIOS does not have SMBIOS version 2.6 or higher and this data, the new naming convention will not be used. Most older hardware does not support this feature because of a lack of BIOSes with the correct SMBIOS version and field information. For BIOS or SMBIOS
  version information, contact your hardware vendor.”

• I don’t know about “less consistent”, but I always considered it a feature in Linux vs the BSDs or big iron Unix that I could always count on the first network interface being “eth0”. BSD and big iron Unix named the interface after the Ethernet driver, as if that was what was important.

  I get that network interfaces can move around on you, but I thought that was why they started putting the MAC address in the ifcfg-eth? scripts.
  What problem did that not solve, that we had to switch to this new system?

  Now I have to remember which *PCI slot* my Ethernet card is in when I
  run “ifconfig” unless I want to dig through the full listing.

  Evil shades of PR#1, begone!

  (Apple DOS 3.3 reference, there.)

• Apparently VirtualBox emulates SMBIOS, since my RHEL 7 VM uses this new scheme.

• conversely, it wasn’t always consistent WHICH NIC would be eth0. Had several x86 servers with dual integral nic’s where eth0/eth1 were swapped relative to what RHEL/CentOS thought they were. while you COULD force it via mucking about in some kernel file or another, it was rarely worth the hassle.

• What does ‘first’ mean? And the same one isn’t consistently first.

  The problem is when you clone a disk and ship it to a location with
  ‘hands-on’ support that doesn’t know linux to install in a new chassis that will arrive there at the same time. Somehow you have to get someone to put the 4 network cables in the right NICs before anything can connect. With things tied to MAC addresses that you don’t know ahead of time, nothing will work.

  Yes, but that’s something you _can_ know.

• Not as dead as you may think, there are still situations where eth0 will be used, even by default:

  [root@el7-test ~]# ip a
  …
  2: eth0: …

  Peter

• Admittedly I like the ethX naming scheme as well, but it did have issues.

  There are situations where the nic needs to be used or at least initialized before networking is brought up, pxe boot comes to mind as just one example. With the new naming convention the nic will have a consistent name everywhere barring any hardware changes (like moving it to a different slot).

  You can still use the old naming convention. There’s a setting somewhere for it and it’s easy to change, I can’t recall off the top of my head where it is, though.

  Peter

• I know the problem you mean, but doesn’t the HWADDR setting in the ifcfg-ethX file fix the problem? Doesn’t that force “ifup eth0” to bind that file’s settings to the right physical interface?

  In the old days, ifcfg-ethX didn’t have HWADDR, so “first” was somewhat unpredictable, as you say.

• So… which PCI/PCI-e slots are associated with the dual gigabit NICs integrated in/on every ASUS board I’ve bought over the last 8 years?

• that forces it to bind to the same interface. but when you’re doing a virgin install, its fairly unlikely you’ll know what the MAC’s are or which port it will pick to be eth0 and the setup will pick them fairly arbitrarily (bus enumeration order, typically)

• How much time and resources do you need to learn the answer?

  Puzzle for ya: What “PCI slot” is the Intel e1000e MAC chip in on a Supermicro X9SCA-F motherboard? It isn’t called out in the mobo manual.
  I just looked. (For that matter, the actual PCI slots don’t have their numbers documented in the manual, either.)

  If you can’t get lucky with Google, you’re just going to have to install EL7 on it and find out. And if you can do that, why not just build it and ship it?

  Yes, I know that problem.

  We solved it here years ago by building the full system, testing it, then labeling the ports with a Sharpie. Then, later, we got really fancy and switched to a Brother label maker.

  Sure, it means we have to have the barebones chassis shipped here first, but as you’re doubtless aware, that shipping charge is cheap next to the confusion that can happen in the field when Joe Wirepuller is asked to plug it all in, if nothing is labeled.

• I wouldn’t know on the first one, but the important thing is that if you have 50 identical servers they would all be the same for the same physical location. The way 6.x works, the motherboard set and the pair on the card will randomly flip in the initial detection. With
  5.x having the MAC address in the ifcfg-ethx file was enough. With
  6.x you also need a udev rule to nail the name down. These get tied to MAC addresses in the initial install, but that makes it painful to clone systems or restore backups into a different box.

• Yes, we’ve had that problem, too. Motherboard model A from Most Favored Manufacturer will put eth0 on the left, then model B replaces it, and eth0 is now on the right. We figure this out during setup, then label the ports.

  If we decide we really want eth0 and eth1 swapped, we swap the HWADDR
  lines between the ifcfg files.

• On top of that, it seems to be only populated slots.

  I added a USB3 PCIe card to my Gigabyte MB and the previous ‘enp2s0’
  became ‘enp3s0’!

• You need a box in a lab setting where you do the build you want to clone.

  Let anaconda figure it out. I don’t care what it is, just that it is repeatable.

  Don’t want to ship the chassis twice – and especially not for the
  2nd/3rd installs on a remote box. I want to send a disk and have someone on-site plug it in and have the box come up working. For the
  2nd/3rd installs, I can get the MAC addresses, but usually don’t know them on the first round.

  It gets old when you are doing several a day. Oh, and we’ve been waiting over a month for a resolution on a server that disappeared in transit, too…

• Oh great, another naming scheme invented by people that don’t actually deal with hardware.

• Awooga! Awoooga! Awooga!

  Here’s the fun part; devices discovered by Anaconda may not match the devices disovered during the production boot. Device driver order and bus discovery order wasn’t necessarily consistent with the production kernel. This is why the HWADDR stuff was added; to work around (poorly)
  this issue. I say poorly becuase I’ve seen many cases of _net#####
  devices where the ifcfg files conflict in same way with the actual device.

  Ultimately what we have is a situation similar to hard disks. We’ve got used to sd devices changing depending on the order disks are discovered in, which is why we use LABEL or UUID. HWADDR doesn’t work consistently.

  The existing process is demonstrably broken.

  The new process is new and therefor bad, wrong, disgusting, an abomination.

  But maybe… just maybe… it’ll work.

• But those don’t work until something has already identified the device. If you are old enough, you might remember unix versions that named disks by controller, bus, target numbers. Which worked, but wasn’t very human-friendly either.

  Maybe. If the names are relative to populated slots, maybe not.

• At install time “we have a disk; we designate it ‘datadisk’ we give it label DATA”. That’s what Anaconda does. The production kernel might find it as another disk, but because it has the label then all works. There’s still a “boot” dependency, but there’s not a lot we can do to work around the BIOS.

  You mean the “modern” c0t0d0s0 type structures (eg Solaris SPARC) and similar
  (truncated) SVR4 Intel paths? Heh, I’m much older than that.

  That was actually not a bad scheme… but it required the bus to be detected in a consistent format. The problem with the Intel architecture is that this detection is _not_ consistent. It depends on module loading order, hotplug device issues etc etc. “c0” isn’t necessarily “c0” on an Intel platform. That’s where it all fell down.

  Back in the day (if you can remember back that far), Dell servers were a fun issue with RedHat; the install kernel would detect devices on the PCI
  bus in one order but the production install kernel would detect them in the _reverse_ order. So if you had two ethernet cards eth0 and eth1 would be reversed between install and boot kernels. Some HP servers also did this. Fun times!

• /dev/rdsk/c0t0n0q0w0e0p1k5n8…. :)

  It’s another reason I took to Linux quickly, right along with eth0.

  Ve shall see.

• I’m old enough to remember another distribution where the trick was to symlink to /dev/cdrom. DO you? :-P

• I agree – it was so much easier when we switched to CentOS from FreeBSD all the interfaces were ethx instead of vrx, rlx, edx, fxpx, emx, bge, etc – it made it so much easier when the hardware platform changed to move our software configs.

• How can you know it? And if you know which port is which why isn’t in the instruction with the drive so the people on site know which port is which?

• Hmm… we have over a 1000 units in the field and CentOS always enumerates the 6 ethx interfaces the same – as they are labeled by the manufacturer of the hardware. This has continued to be consistent even when the manufacturer upgraded the MB.

• What do you mean, “slot”? All of my servers, and our systems at home, the NIC’s on the m/b. What “slot” is that? Is it labeled *anywhere*? No, of course not.

  mark

• Many servers have PCI cards for NICs in addition to those on the motherboard (if any). For example, most of my file servers have eight ethernet interfaces (six 1GBE, two 10GbE). On my Dell servers, the built-in interfaces are labeled on the back panel.

  However, at least in CentOS 6, you can call the interfaces anything you want by suitably changing /etc/udev/rules.d/70-persistent-net.rules. The names used have to be consistent with /etc/sysconfig/network-scripts/ifcfg-*
  of course.

  BTW, I have some workstations that have only a single interface, and that comes up as p2p1. I actually like the new scheme better, but don’t get me started on the use of UUID in /etc/fstab…

  Steve

• Are these Dells using the bios name conventions? Or maybe it works better if all NICs are the same vendor/model. Ours mostly have onboard Broadcomms’s and Intel cards, and they routinely flip pairs of interfaces – the sets on the motherboard and on any particular card will stay in the same order, but the order of the motherboard set and any card will be random during the install. Worse, when you move a drive to a different chassis, the old udev rules keep any of the new set from matching any old name.

• If you insert the card yourself, you obviously know the slot. And you can tell the position from the back just by looking at it. But CentOS6 will detect in random order, so knowing the name on one box doesn’t help with another. We have to go through contortions plugging on cable in at a time, doing an ‘ifconfig up’ and checking which interface shows link up. And the people doing that part wish we used more windows instead of Linux.

• How about using ethtool -p which causes the LED of the NIC to blink?

• Les Mikesell wrote:
  

  When you clone a disk, you can’t get the ifcfg-eth* and
  70-persistant-net-rules from the old machine, or you don’t have that info under version control, with all those systems?
  

  ifconfig up? Not ethtool eth?

  mark

• Am Wed, 15 Jan 2014 16:25:04 +0200
  schrieb JC Putter :

  Very useful, unless the datacenter isn’t in the basement ;-)

• Steve Thompson wrote:
  No, of

  I can think of one server we’ve got, a Dell PER815, that’s got a NIC (that we don’t use, dunno why it was in there), and four onboard. I disliked it when the first time I started doing sysadmin, on a Sparcserver 20, back in the mid-nineties, and I don’t like it any better now. Among other things, too easy to mistype one of the letters or numbers.

  About UUIDs, though, I think we can start on that in harmony. UUID is *so*
  much easier to remember, and shorter than, say, the serial number on the disk label… oh, right, it’s twice as long….

  mark

• I don’t have that information on the 1st install. Once it is up and running the ocsinventory-ng client will report the hardware to a central server. I suppose with a huge increase in human effort needed, the MAC addresses of the systems and cards could be databased as they are installed.. But, even knowing them doesn’t help when you clone multiple identical disk images. And something would still have to map the MACs to the physical positions.

  You have to do both. Link won’t come up until you ifconfig up the device – which of course is difficult when you don’t know its name…

• Les Mikesell wrote:
  

  I don’t think so – pretty sure I was just using ethtool eth? the other week, to try to figure out the name of the port that I’d plugged the patch cord into. I *know* that the ones with nothing in them weren’t up (and yes, obviously, I was at the console).

  mark

• Maybe something else had already probed them. I’m pretty sure that if you bring up a system where all of the udev rules and ifcfg- files have the wrong MAC addresses, none of the links will come up. With CentOS5 you could use mii-tool to enumerate the interfaces and show link. I think the best I’ve found with 6 is to use ‘ip link ls’ to show the names, then one at a time ‘ifconfig up’ each name and then use ethtool to check for link. All very awkward to describe to a windows admin over the phone.

• I know what you mean, but on those systems, the Ethernet MAC chip is still on the PCI[e] bus, so it still gets a slot number. If you say lspci, it’s generally the second number, between the colon and dot.

  Fun fact: the MAC chip isn’t always on a PCI[e] bus. On the Raspberry Pi, it’s on the USB bus, despite the fact that it’s soldered to the PCB!