Rsyslog Does Not Log On A Separate Partition/FS Mounted On /var/log/

Home » CentOS » Rsyslog Does Not Log On A Separate Partition/FS Mounted On /var/log/
CentOS 8 Comments

The system is an AWS Instance based on a community CentOS 6.4 AMI snapshot.

The vdisk is as follows as shown below [1]
The root LVM contains /var/log/

I have attached another block device with ext4 FS.

I copied the files from /var/log to this device (mounted on /mnt) and then changed
/etc/fstab to mount this device on /var/log on boot.

However, I do not see anything being logged in /var/log/messages. To test the logging, I used the ‘logger’ command to log some string; nothing appears in /var/log/messages.

‘service rsyslog status’ reports the daemon is running.

When I stop rsyslog, umount the /var/log device and then restart rsyslog, I can see that logs are being recorded in /var/log/messages. Using the ‘logger’
command I can see messages written in /var/log/messages.

man pages of ryslog.conf and rsyslogd show nothing related to logs being on a separate device

Any pointers to fix the problem would be much appreciated.

[1]
Disk /dev/xvda: 8589 MB, 8589934592 bytes

Device Boot Start End Blocks Id System
/dev/xvda1 * 1 64 512000 83 Linux
/dev/xvda2 64 1045 7875584 8e Linux LVM

Disk /dev/mapper/VolGroup-lv_root: 7205 MB, 7205814272 bytes Disk /dev/mapper/VolGroup-lv_swap: 855 MB, 855638016 bytes

— Arun Khan

8 thoughts on - Rsyslog Does Not Log On A Separate Partition/FS Mounted On /var/log/

  • In article , Arun Khan wrote:

    Probably rsyslog is being started before /var/log is mounted, and so it is opening files within /var/log on the root device.

    When the second device gets mounted on /var/log, the files within the original /var/log are no longer visible, but rsyslog still has open handles to them.

    You need to arrange for rsyslog to get restarted or HUPed after the mounting of /var/log.

    Cheers Tony

  • rsyslog should start after local mounts are finished.

    I suspect it’s selinux; /var/log should have a “var_log_t” context and I
    suspect it doesn’t.

    John

  • In article <20140806165735.GD10943@frodo.gerdesas.com>, John R. Dennison wrote:

    Ah, ok, thanks. I hadn’t actually gone and looked…

    Be interesting to know if that fixes it for the OP.

    Cheers Tony

  • But would that explain why, when the OP umounts /var/log, the latest logs have been written to /var/log/messages on the / filesystem?

    It certainly can’t hurt to check both cases: make sure rsyslog is starting after the proper filesystem with /var/log is mounted, and check the selinux contexts to make sure they’re correct.

    –keith

  • rsyslog is started with a start priority of 12; long after
    /etc/rc.d/rc.sysinit finished mounting local filesystems. Syslog’s initial ring-buffer is dumped nearly at the end of sysinit so that should be getting dumped to the newly-mounted /var/log as well.

    Kind of curious as to why that is not happening.

    John

  • SOLVED

    I had suspected SElinux and have it disabled still rsyslogd was not logging on the new device mounted on /var/log/

    *** restorecon -vv /var/log does the trick! ***

    @ James A. Peltier Thank you!

    FWIW – here are the steps

    1. service rsyslog stop
    2. mount /mnt/
    3. rsync -aP /var/log/ /mnt/
    4. rm -fr /var/log/*
    5. umount /mnt
    6. mount     /var/log/ (also make change to /etc/fstab)
    7. restorecon -vv /var/log <<< the solution 8. service rsyslog start. 9. logger "this is a test" 10. tail /var/log/messages to verify that indeed the logger string was logged. — Arun Khan

  • If restorecon fixes the problem, then you never disabled SELinux

    If you untar files into a location, you should always run restorecon on the directory to fix the SELinux labels.