SEtroubleshootd Crashing

Home » CentOS » SEtroubleshootd Crashing
CentOS 11 Comments

When running Node.js through Phusion Passenger on CentOS 6.5 ( Linux 2.6.32-431.23.3.el6.x86_64 #1 SMP Thu Jul 31 17:20:51 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux), with SELinux enabled in permissive mode we receive a large number of entries in the audit.log and setroubleshootd randomly crashes with the following error, We have resolved the selinux alerts by following the troubleshooting steps recommend by running sealert,However we are concerned by setroubleshootd crashing and are concered that we may have masked the issue by fixing the entries in the audit.log.

abrt_version: 2.0.8

cmdline: /usr/bin/python -Es /usr/sbin/setroubleshootd -f ”

executable: /usr/sbin/setroubleshootd

kernel: 2.6.32-431.23.3.el6.x86_64

last_occurrence: 1417101625

time: Thu 27 Nov 2014 03:20:25 PM UTC

uid: 0

username: root

sosreport.tar.xz: Binary file, 3642240 bytes

backtrace:

:analyze.py:426:lookup_signature:ProgramError: [Errno 1001] signature not found

:

:Traceback (most recent call last):

: File “/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py”, line 401, in auto_save_callback

: self.save()

: File “/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py”, line 377, in save

: self.prune()

: File “/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py”, line 340, in prune

: self.delete_signature(sig, prune=True)

: File “/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py”, line 471, in delete_signature

: siginfo = self.lookup_signature(sig)

: File “/usr/lib64/python2.6/site-packages/setroubleshoot/analyze.py”, line 426, in lookup_signature

: raise ProgramError(ERR_NO_SIGNATURE_MATCH)

:ProgramError: [Errno 1001] signature not found

:

:Local variables in innermost frame:

:matches: []

:siginfo: None

:self:

:sig:

We are running the following versions Passenger/htttpd/node

passenger –version

Phusion Passenger version 4.0.53

httpd -v Server version: Apache/2.2.15 (Unix)
Server built: Jul 23 2014 14:17:29

node -v v0.10.32

This email is from the Press Association. For more information, see www.pressassociation.com. This email may contain confidential information. Only the addressee is permitted to read, copy, distribute or otherwise use this email or any attachments. If you have received it in error, please contact the sender immediately. Any opinion expressed in this email is personal to the sender and may not reflect the opinion of the Press Association. Any email reply to this address may be subject to interception or monitoring for operational reasons or for lawful business practices.

11 thoughts on - SEtroubleshootd Crashing

  • Thanks

    Could you please clarify, which version libxml is broken and has there been a newer version released that will fix it.

    —–Original Message—

  • I am not sure. I was just seeing email on this today. Could you try to downgrade the latest version of libxml to see if the problem goes away.

  • We are currently running libxml2-2.7.6-14.el6_5.2.x86_64

    How far back would you suggest we go? would libxml2-2.7.6-14.el6_5.1.x86_64 be sufficient

    —–Original Message—

  • Ok might not be related. One other suggestion would be to clear the database out. And see if there was something in the database that was causing it problems.

    Make sure there is no setroubleshootd running and

  • I’ll jump in here to say we’ll try your suggestion, but I guess what’s not been mentioned is that we get the setroubleshoot abrt’s only a few times a day, but we’re getting 10000s of setroubleshoot messages in
    /var/log/messages a day.

    e.g.

    Dec 2 10:03:55 server audispd: queue is full – dropping event Dec 2 10:04:00 server audispd: last message repeated 199 times Dec 2 10:04:00 server rsyslogd-2177: imuxsock begins to drop messages from pid 5967 due to rate-limiting Dec 2 10:04:01 server rsyslogd-2177: imuxsock lost 2 messages from pid
    5967 due to rate-limiting Dec 2 10:04:01 server audispd: queue is full – dropping event Dec 2 10:04:02 server audispd: last message repeated 134 times Dec 2 10:04:02 server setroubleshoot: SELinux is preventing /bin/ps from read access on the file /proc//stat. For complete SELinux messages. run sealert -l 2274b1c7-fd69-4fa8-8e67-cd7a9da9eff4
    Dec 2 10:04:02 server audispd: queue is full – dropping event Dec 2 10:04:03 server audispd: last message repeated 48 times Dec 2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps from getattr access on the directory /proc/
    . For complete SELinux messages. run sealert -l 2d09d555-8834-4c27-976b-6647f8673286
    Dec 2 10:04:03 server audispd: queue is full – dropping event Dec 2 10:04:03 server audispd: last message repeated 15 times Dec 2 10:04:03 server rsyslogd-2177: imuxsock begins to drop messages from pid 5967 due to rate-limiting Dec 2 10:04:03 server setroubleshoot: SELinux is preventing /bin/ps from search access on the directory /proc/
    /stat. For complete SELinux messages. run sealert -l 0ef0c7a1-acb2-433a-aaa2-361cc95b6069
    Dec 2 10:04:04 server setroubleshoot: last message repeated 2 times Dec 2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps from getattr access on the directory /proc/
    . For complete SELinux messages. run sealert -l 58f859b0-7382-428e-81f0-3e85f66d79fc Dec 2 10:04:04 server setroubleshoot: SELinux is preventing /bin/ps from search access on the directory /proc//stat. For complete SELinux messages. run sealert -l 2448a46d-5089-4f85-aae8-e9013341471f Dec 2 10:04:05 server setroubleshoot: last message repeated 2 times Dec 2 10:04:05 server setroubleshoot: SELinux is preventing /bin/ps from getattr access on the directory /proc/. For complete SELinux messages. run sealert -l f935416b-54fe-4bbd-b66c-2e1b2e6724be Dec 2 10:04:06 server setroubleshoot: SELinux is preventing /bin/ps from search access on the directory /proc//stat. For complete SELinux messages. run sealert -l d8dbf973-7bc2-4fd5-9540-18c4040be03c Dec 2 10:04:06 server setroubleshoot: last message repeated 2 times Dec 2 10:04:06 server sedispatch: AVC Message for setroubleshoot, dropping message Dec 2 10:04:06 server sedispatch: last message repeated 3 times

    Cheers,

    John

  • Could you send me a copy of your audit.log.

    You should not be getting hundreds of AVC’s a day.

    ausearch -m avc,user_avc -ts today

  • Mark: Labels look OK, restorecon has nothing to do, and:

    -rwxr-xr-x. root root system_u:object_r:bin_t:s0 /bin/ps

    dr-xr-xr-x. root root system_u:object_r:proc_t:s0 /proc

    I’ll send the audit log on to Dan.

    Cheers,

    John

  • Looks like turning on three booleans will solve most of the problem.

    httpd_execmem, httpd_run_stickshift, allow_httpd_anon_write

  • Indeed, thanks Dan – it doesn’t get us to a completely clean running that would allow us to run our Node app as we are under Passenger with SELinux enforcing, but it at least has stopped the excessive amount of AVCs we were getting.

    John