Hacker News new | past | comments | ask | show | jobs | submit login

Using DAQ's properly eliminate process blocking when using TCP delivery and there are connectivity issues. You can do TLS encryption natively. Rsyslog action templates are complex but extremely powerful. After reading this post I just wonder if you have ever read the Rsyslog documentation or done any large scale deployments with it? Our network is handling over 50k log messages a second using Rsyslog and while it's not perfect I can't think of any other standards based system I could rely on.



Hi, author here :-)

We do use rsyslog.. The rates we see aren't terribly high, but it is a fairly large deployment. We have a setup consisting of redundant tcp load balancers fronting redundant syslog relays feeding into additional applications and an archival box.

Basically this example in the RFC, but with a frontend load balancer cluster and additional collectors:

   +----------+         +-----+            +---------+
   |Originator|---->----|Relay|---->-------|Collector|
   |          |-+       +-----+        +---|         |
   +----------+  \                    /    +---------+
                  \     +-----+      /
                   +->--|Relay|-->--/
                        +-----+
For the pieces that we control and can use RELP things work great. The problem is that we run this as a service for a large group of heterogeneous systems. We split out logs by hostname, and it's not uncommon to wind up with an INFO.log at the end of the day because someone is sending us INFO where the hostname should be.

If every client could use rsyslog+relp and things like imfile to send us application logs, the whole system would work a lot better.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: