Hacker News Reader

That works fine until your centralized server falls over, either because you're scaling to hundreds or thousands of machines and your logs are chatty or someone goofs and starts logging kilobytes of data on every request.

View on HN

If you care about that, it's very easy to have redundancy there. In fact, the target pipe you route to on each container/system/etc. can be fully virtualized and routed as needed by... syslog. ;-) If you want to go crazy you can wrap it in rate limiting which is done for you by... syslog.

There would be usually multiple syslog sink servers so there is never a single point of failure. This can be configured in various ways either dns round robin, load balancers or even multiple destinations configured on the client side, rsyslog can iterate over a list in case of failures.

If there is log storm coming from misconfigured app depending on your traffic levels the sink servers should give you a lot of space for an on-call engineer to be notified that something is off and adjust / rate limit / drop accordingly.

Edit: udated with more info

Reply on news.ycombinator.com