How To Know When A System Is Compromised

Home » CentOS » How To Know When A System Is Compromised
CentOS 8 Comments

How do you know when a Linux system has been compromised? 

Every day I watch our systems with all the typical tools, ps, top, who, I watch firewall / IPS logs, I have logwatch setup and mailing daily summaries to me and I dive deeper into logs if something looks suspicious.

What am I missing or not looking at that you security gurus are looking at?

I subscribe to the CentOS and SANS newsletters, and I try to keep current on all technology with credible sources of articles online and with the Lynda library.

What other sources of information do you use to stay current about the latest threats and technology updates?

I appreciate the feedback.

Chris

8 thoughts on - How To Know When A System Is Compromised

  • This is one where there’s probably no limit to what you could do. We have a high-security environment and are using Aide and OSSEC.

    Aide has been good at reporting file system changes and is very granular, the dilemma is what to monitor and what to ignore (keep from being inundated with reports of innocuous changes at the risk of missing something). However, it is not daemon-based so changes between runs which are undone go unnoticed. Also, somehow you need to protect the executable and configuration file so that an attacker can’t replace the executable or read the configuration and find a way around it. The executable could be placed on mounted read-only media, last time I checked Netac and Kanguru still made USB sticks with write-protect switches. Our best effort for protecting configuration is to deliver the configuration file just-in-time and delete it after the scheduled run, not a great solution, anybody have a better idea?

    OSSEC is daemon-based and centrally-managed. It is a HIDS rather than just a FIMS as Aide is. Its log monitoring has surfaced operational issues in addition to security ones (Postfix got in an odd state and had to be restarted for example). Unfortunately, false positives are common, especially if you use the “detect new files” feature. They admit that dealing with software updates is problematic.

    I’ve used auditd to trace down what ended up being a funny situation, Aide detected that /etc/hosts.deny would change timestamp but nothing else, turns out OSSEC has an active response feature to block attacks which involves updating that file to block a host for 10 minutes.

    You could also look into inotify options and Samhain is another HIDS (I’d love to hear about anyone’s experience with it). A free variant of tripwire may still exist but is probably unsupported and Aide is a clone of it.

    I noticed that rootkit detection has also been mentioned in another reply.

  • I’m sure you have followed the procedure how to install system and services so everything is secure.

    If, in a longer run no matter that you have system set up and configured securely and keep updating, if still the system gets compromised, then you need:

    1. compromise warming
    2. forensic investigation
    3. recovery from compromise.

    I figure your is about 1. You probably will not get detailed description of actual setup people on this list have. Information about what the defense is is the first step in every attack. The best you may get are the advises of what to look for.

    One of the things you can set up is [host based, maybe] system integrity checking system (or intrusion detection system). That only makes sense on freshly installed system in known good state. There were a variety of these: tripwire (which went commercial), eics, … If you search for linux intrusion detection system you should find what you need.

    I hope, this helps.

    Valeri

  • Once upon a time, Leroy Tennison said:

    That’s not as secure as you think. Linux bind mounts can mount a file over another file (plus there’s overlay filesystems), so it’s possible to replace a binary even on a read-only device.

  • Actually, a defense here is to umount the path then remount it as a part of running the Aide script. There may be an end-run to this as well- security is a never-ending battle.