Centralising logs with rsyslog and parsing them with Graylog extractors

brendan.abolivier.bzh

2 points·babolivier··4 comments

Parsing this way (in Graylog) is so very... manual. I'd rather go with rsyslog's liblognorm, either in rsyslog itself or in a separate daemon, like logdevourer I wrote. The rules then can be written in a regular text editor and versioned.

Also: what happens if there is a typo in one of the rules and a log entry doesn't get parsed? Does Graylog/grok have some indication that the log doesn't match? And where the mismatch starts? Because liblognorm does.

I'm not saying that's the best way to do it, just that it's how we did it back then. Of course there might be better ways to do it. I was in charge of exploring this way (which we didn't end up keeping, as I hint at at the end of the post), and thought it would be valuable to share what I learned with people.

Regarding your latter point, as I mentionned, a pattern fails silently if it doesn't match a log entry while parsing incoming entries. That's why Graylog requires you to load a log entry example when creating an extractor, and let you try it while defining the pattern (with the "Try" button next to the pattern input field). I didn't put enough emphasis on this feature in my post, though.

> That's why Graylog requires you to load a log entry example when creating an extractor

So it still fails silently when the log format changes slightly, e.g. upstream fixes spelling or grammar in one of the log messages produced.

Yep. You might be able to configure Graylog to alert you when an extractor fails, but I didn't go this far (as I said, we switched to another solution since then).

Reply on news.ycombinator.com