Systemd Debug Logging Turned On In CentOS 7

Home » CentOS » Systemd Debug Logging Turned On In CentOS 7
CentOS 8 Comments

Hi,

I noticed that some, but not all, of my CentOS 7 machines have these kernel parameters for turning on systemd debug level logging added to the grub.cfg file.

systemd.log_level

8 thoughts on - Systemd Debug Logging Turned On In CentOS 7

  • Oh wow, I was coming across this in my env last week and thought that it was something with the way they were kickstarted. You are not alone!

  • Once upon a time, Thomas Eriksson said:

    Yep, for each of the installed kernels, I have two GRUB entries: one with and one without debugging. It seems that when I install a new kernel, the debugging entry gets the “default” choice.

    I don’t know what’s adding them though.

  • Last time I saw it, I had just upgraded my CentOS 7 box with the 3.10.0-514 kernel and it rebooted already configured into debug mode. Not sure if this is a “feature” of the newer kernels or not but glad to see that i’m not the only one who had noticed this.

    # awk -F\’ ‘$1==”menuentry ” {print i++ ” : ” $2}’ /etc/grub2.cfg
    0 : CentOS Linux (3.10.0-514.6.2.el7.x86_64) 7 (Core)
    1 : CentOS Linux (3.10.0-514.6.2.el7.x86_64) 7 (Core) with debugging
    2 : CentOS Linux (0-rescue-7b37bcbe36eb420fb6426976c41b0aaf) 7 (Core)
    3 : CentOS Linux (0-rescue-7b37bcbe36eb420fb6426976c41b0aaf) 7 (Core) with debugging

  • I am not certain if there is real harm to have kernel with all debug stuff running on production machines. Probably no harm security wise, the only unpleasant stuff is: you really would prefer to run as slim kernel as possible on production systems. If I’m wrong about “no harm”, somebody chime in, I then will be really eager to address it on my boxes.

    Valeri

    ++++++++++++++++++++++++++++++++++++++++
    Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247
    ++++++++++++++++++++++++++++++++++++++++

  • The reason why I noticed was that telegraf was reporting context related issues even with selinux disabled. That’s when I noticed all of the debugging lines in dmesg as well as debug mode enabled in /proc/cmdline.

  • Main issue I’ve seen is that logs grow by an order of magnitude larger than when it’s off, due to systemd being systemd and now running in debug mode. Other than disk space, it would affect any central logging system you have with lots of unnecessary traffic, and would also add a lot of IO, amplified if you have many machines running on a VM host.

    ~ Brian Mathis
    @orev

  • Just to put the record straight; it’s not related to kernel debugging being enabled or not. It’s systemd debugging that is being turned on for all menu entries, kernel debug or not.

    Anyway, I think I have found a pattern. Only those machines that were updated from 7.2 to 7.3 using the CR repo are showing this behaviour. New 7.3 installs are fine. I’ll just clean up the machines affected and move on. There must have been some debug config left in an installation script in one of the CR rpms.

    thanks to everyone responding,

    Thomas

  • Thomas

    Yes, true, this isn’t technically the “kernel” debug mode, but systemd debugging is also enabled for the debug boot options, which seems to get carried into a non-debug boot entry somehow.

    I have seen this issue on a few machines, and I don’t use the CR repos. The ones I saw were during upgrades from 7.2 systems with all updates, which were then upgraded to 7.3 with all updates as of 2 weeks ago.

    ~ Brian Mathis
    @orev