Systemd

Home » CentOS » Systemd
CentOS 30 Comments

So I’ve built my first CentOS 7 host and am learning all the new ways of doing things. I setup and enabled ntpd but after a reboot I get:

$ systemctl status ntpd ntpd.service – Network Time Service
Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
Active: inactive (dead)

After issuing:

$ systemctl start ntpd

Ntpd runs just fine. But why isn’t it loading at boot time?

30 thoughts on - Systemd

  • El sáb., may. 23, 2015 10:37 PM, Kirk Bocek escribió:

    …

    Ntpd runs just fine. But why isn’t it loading at

    boot…

    Did you run

    systemctl enable ntp.service

    after installing it?

    HTH, Carlos.

  • Yup, take a look at the second line:

    $ systemctl status ntpd ntpd.service – Network Time Service
    Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
    Active: inactive (dead)

  • So reviewing Red Hat’s documentation on the subject:

    https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/System_Administrators_Guide/chap-Configuring_the_Date_and_Time.html#sect-Configuring_the_Date_and_Time-timedatectl-NTP

    I see both chrony and ntpd mentioned but no preference given. Seeing that I’ve used ntpd for a decade and am more comfortable an familiar with it, I think I just stick with ntpd.

    However, we have yet *another* new control system. We are supposed to issue:

    $timedatectl set-ntp yes

    to activate your selected daemon. I just used the new systemd commands, thinking that would be enough. So I tried that and rebooted. Nope, same problem:

    $systemctl status ntpd ntpd.service – Network Time Service
    Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
    Active: inactive (dead)

    I have also tried:

    $systemctl reenable ntpd

    To get the symbolic links rewritten. No help.

    To be clear, if I manually issue:

    $systemctl start ntpd

    ntpd runs fine:

    $systemctl status ntpd ntpd.service – Network Time Service
    Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
    Active: active (running) since Sun 2015-05-24 11:39:21 PDT; 8s ago
    Process: 2935 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS
    (code=exited, status=0/SUCCESS)
    Main PID: 2936 (ntpd)
    CGroup: /system.slice/ntpd.service

  • Every CentOS 7 installation that I’ve done so far installs and activates chrony by default, without any particular action required on my part at all.

  • chronyd and ntpd both use UDP port 123, so each will terminate the other when it starts. If both are enabled, chronyd’s unit file indicates that it should start after ntpd, so it will always “win.”

    You probably have both enabled, so the system boots, starts ntpd, then starts chronyd which terminates ntpd.

    Disable chronyd.

  • Yes, indeedy.

    $systemctl status chronyd chronyd.service – NTP client/server
    Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled)
    Active: inactive (dead) since Sun 2015-05-24 11:39:21 PDT; 3h 31min ago
    Process: 845 ExecStartPost=/usr/libexec/chrony-helper add-dhclient-servers (code=exited, status=0/SUCCESS)
    Process: 808 ExecStart=/usr/sbin/chronyd -u chrony $OPTIONS
    (code=exited, status=0/SUCCESS)
    Main PID: 814 (code=exited, status=0/SUCCESS)
    CGroup: /system.slice/chronyd.service

    May 24 11:35:53 cobalt chronyd[814]: NTP packet received from unauthorised host 10.0.0.1 port 123
    …

    So:

    $rpm -e –nodeps chrony

    chrony has dependencies with anaconda and initial-setup. Reboot, and:

    $systemctl status ntpd ntpd.service – Network Time Service
    Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled)
    Active: active (running) since Sun 2015-05-24 15:15:42 PDT; 3min 1s ago
    Process: 847 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS
    (code=exited, status=0/SUCCESS)
    Main PID: 868 (ntpd)
    CGroup: /system.slice/ntpd.service

  • Chrony is actually a useful tool… years and years ago, when Chrony was young and being maintained by the original author, I ran it here at home, because I had dialup only. Ran it on a (even then) obsolete Pentium 90 box that also ran the GPL version of Smoothwall. Configured PPP to autodial when someone wanted to access the internet, and we’re off to the races. Chrony kept the smoothwall box fairly accurate, and all the system on the LAN then synced with the smoothwall system.

    Chrony is actually designed to be able to work properly on such systems/
    networks, where internet connectivity is intermittent, and to do a good job of keeping the time within a reasonably near facsimile of actual time.

  • No. Bad.

    Just disable the service. Breaking your rpm database will just lead to pain down the road. Disabling the service will maintain the integrity of the package dependencies, and most likely a later yum update will reinstall chrony, possibly enabled.

  • No, not that I can tell…

    # rpm -q –whatrequires chrony

    Either way, don’t use –nodeps. Just don’t. If you break dependencies, you’re going to have trouble updating later. Or something unexpected will stop working.

    In this case it’s not a big deal, since nothing actually requires chrony, but in general it’s a really bad practice.

  • Okay, okay! I’ll go on the paper.

    I’ll reinstall chrony. But there *are* places I’ve needed to use nodeps. Mostly to manage inter-repo package incompatibilities.

    Fortunately this isn’t yet a production host but more a chance to learn all the changes like this, systemd, grub2, xfs, whatever the new firewall is called. The list keeps growing.

  • You should look into the yum priorities option to ensure packages from different repos don’t step on each other, however I’d consider any yum repository that requires you break your rpm database to be one to avoid.

  • It usually happens when I’ve wanted to change versions of the same software between repos and that software has been compiled differently. Usually audio-visual software. Yum install triggers a conflict and yum uninstall on the older package cascades a bunch of undesirable uninstalls. The only solution is rpm -e -nodeps followed by installing the newer package.

    If you have a better solution, let me know.

  • The need to use –nodeps undermines the idea of rpm based (or more generally package based) system maintained in harmony by system vendor
    (and extra repository vendors). Basically, you become individual system vendor for yourself, which is a lot of work. If it comes to that, then you will be in much better shape (from the point of view of how much time you spend on keeping your box in harmony with all components working) if you switch from package management based system to source build based system. Examples of the last just off the top of my head would be: gentoo linux, FreeBSD, NetBSD… The last two are sort of “port based systems” as far as extra software is concerned. (Of course, BSD boxes in addition to ports have package management, so in their case you can maintain system either way: as prebuilt packages, or as building ports yourself, the last gives you ability to change configuration options of what you build to your specific needs).

    Not intending to criticize what you are doing, just giving you an idea what it looks like for external observer.

    Just my $0.02

    Valeri

    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • If the new package fulfills all requirements as the old one, you can use
    “yum shell” to do both transactions at the same time.

    $> yum shell

    -Thomas

  • Stop using those repos. Whatever you’re doing, you’re not using a repo or software packaged for the version of CentOS you are using. Could you give some examples? It would help to see these dependency mismatches in person.

    If you want to use the software, find the source RPMs and rebuild them for the version of CentOS you’re using.

    —
    Jonathan Billings

  • I am an avid user of MythTV and prefer to install from RPM. There are
    *very* few sources of those RPMS. ATRpms used to be my main source:

    http://packages.atrpms.net/dist/

    But Axel Thimm seems to have drifted off to better things and he stopped building the latest versions of Myth. SCRpms provided by Stephen Collier has picked up the torch:

    http://scrpms.net/pub/

    Now the problem has been less these repos than the host of supporting audio-visual packages needed to get MythTV up and running: things like FFMpeg, the latest proprietary Nvidia driver, lirc. Often I have to go to third party repos to get things working.

    One example of the conflicts involved here are the QT packages. It looks like CentOS 7 ships with QT 4.8 which is what MythTV currently requires. But CentOS 6 shipped with QT 4.6. Stephen Collier did a really good job compiling his 4.8 packages for CentOS 6 so they could install in parallel. *Except* for the qt-x11 package. That one cannot be installed in parallel. If you have anything depending on qt-x11, you’ll have to rpm -e –nodeps the 4.6 package before installing the 4.8 package. The regular yum upgrade process somehow doesn’t work. As I recall it wants to pull in a bunch of other stuff and conflicts arise.

    The bigger issue is a project like MythTV being targeted at the bleeding edge like Fedora while I want to stay on the stable edge with CentOS. I’ve had to deal with this for years. MythTV will eventually move on to a library or a tool not supported by the base CentOS install and it will be a battle to get it to work.

  • Hi List,

    I am trying to understand what After= means in a unit file. Does it mean after the specified target is up and operational or only that the target has been started?

    I have something that needs postgres but postgres needs to be operational not just started. Sometimes it can take a bit for postgres to become operational.

    Thanks, Steve

  • I believe that the PostgreSQL service has Type=notify in it’s service definition, which means that it will notify systemd when it is operational. This means that if you have a service that has After=PostgreSQL.service, systemd should wait until after the PostgreSQL service notifies systemd that it is operational before your service will be started.

    If your service is starting and unable to connect to PostgreSQL, then I would say that’s a bug in PostgreSQL — it shouldn’t be notifying systemd that it is operational until it actually is.

  • This is, in fact, one of the points why I’m very unhappy with systemd and the way it is implemented here and most likely in most distributions.

    Maybe things _could_ be done the right way with systemd, but it doesn’t happen because it quickly starts to be very complex and it’s a lot of work to do it for a complete distribution. It just doesn’t happen – or at least did not happen in all the years since its introduction.

    In this example, PG gets just started with “pg_ctl start” and that’s it.

    Regards, Simon

  • Hmm… I don’t see that in the PostgreSQL.service file – this is CentOS Linux release 7.5.1804 (Core)
    PostgreSQL-server-9.2.24-1.el7_5.x86_64

    from /usr/lib/systemd/system/PostgreSQL.service
    …
    [Service]
    Type=forking

    User=postgres Group=postgres
    …

    Regards, Steve

  • Yes, introduction of systemd earned Linuxes a lot of refugees. I in my worst times feel maybe that was the goal of it. But then I think about a split of refugees from Linux to UNIX descendants (FreeBSD, NettBSD etc.)
    vs to MS products, and I am not quite certain if that was a goal (though I do remember MS alliance with RedHat…), but if it was the goal I
    doubt refugee split was in MS favor (though one says something is better than nothing).

    I hope, this didn’t come as a rant, I should probably have used rant tags ;-)

    Valeri

  • You’re right! My mistake.

    I was looking at the systemd service for PostgreSQL 10 (check out the rh-PostgreSQL10-PostgreSQL-server package in SCL). It seems that they’ve managed to get sd_notify notification working in version 10, but it’s still using Type=forking in version 9.

    By the way, this isn’t really a systemd issue — even with sysvinit you’d be stuck trying to figure out when the service was *really* up in a shell script or something. At least now there’s a mechanism to tell the startup service that the service has actually started, so proper ordering of services can be automatically performed, rather than stringing together a collection of shell scripts.

  • If you’ve looked at the sysv init script for PostgreSQL, you know that that statement describes both init systems.

    Systems engineering is hard.  It’s fashionable to blame systemd, but it’s not systemd’s fault that there’s a delay between the point at which PostgreSQL forks and the point at which it’s available for use.  SysV
    didn’t magically solve that problem. Someone had to specifically write a delay loop in the init script to make the system work reliably, beforehand.  PostgreSQL isn’t alone in that.  Other services needed their own hacks.  And collectively, “Maybe things _could_ be done the right way with SysV, but it doesn’t happen because it quickly starts to be very complex and it’s a lot of work to do it for a complete distribution.”

  • There are a couple ways that systemd can handle service startup in a way that dependent services can gracefully start up after it. One way is to have systemd open the socket, then hand it to the service when it is ready. This requires quite a bit of hacking and I don’t think is as reliable, it’s more of the inetd way of doing things.

    Another is to have the code send a message when it is ready. This isn’t really that complicated, you can look at the change in PostgreSQL’s git here:

    https://git.PostgreSQL.org/gitweb/?p=PostgreSQL.git;a=commitdiff;h}17e683fcc28a1b371c7dd02935728cd2cbf9bf

    Basically, when the database is ready, it calls the C function:
    sd_notify(0, “READY=1”);

    and when it’s shutting down, it runs:
    sd_notify(0, “STOPPING=1”);

    To be honest, that’s not too complicated. It does require minor changes to the code to support systemd, but you can replace idle loops in shell scripts with a smarter database (which knows when it is ready) telling PID 1 that it is ready.

  • Hi All – I have a systemd service file to start my application.

    Part of my service file is:
    [Service]
    Type=forking ExecStart=/path to start ExecStop=/path to stop RemainAfterExit=yes

    This works fine “normally”.

    However – when I do a “yum update; and reboot” my script detects a kernel change and now does a number of “recompiling” steps for drivers and such before starting my application. Somewhere along the way that just stops. I’m half way into recompiling those drivers and it just stops.

    Is there something else I need in the service file ?

    Thanks

    Jerry

  • Jerry Geis wrote:

    Could it be you are hitting some default timeout ?

    Maybe adding ‘TimeoutSec=infinity’ could help ?

    Or, maybe moving your ‘recompiling’ stage to an ExecStartPre script ?

    (no idea if either of the above will help)

    James Pearson