TL;DR:

Is there a sensible way to get rsyslog to talk to redis on CentOS 7. The official way is to use the omhiredis plugin, but that doesn’t seem to exist in the CentOS 7 repositories? (It’s how I do it on my Fedora box.)

The long version:

I’m trying to rationalise logging and am using an ELK stack. When I’ve done this in the past it has suffered badly from congestion at peak times with messages being dropped and various issues on the clients. The sensible thing seemed to be putting redis between the clients and ELK – that works in my test setup from a Fedora box.

My problem though is ingesting the logs from my CentOS servers. By far the easiest thing is to get rsyslog to send things directly to redis, but that needs the omhiredis plugin.

The second choice is to use filebeat to scan the log files – but if I
want to have things like facility and severity logged, I have to use custom rsyslog templates and it’s a pain trying to get filebeat to parse them (and the filebeat redis output seems a bit flaky as well).

I can see that syslog-ng has redis support, but I would prefer to keep rsyslog since that is the default.

So, what do people do to ship their logs to ELK (or, specifically, logstash). Is there some magic, neat solution? Or do I keep just hammering away at filebeat -> redis -> logstash until I get something that’s reliable? Or do I go down the syslog-ng route?

P.