Plaintext is not a great format for (system) logs | Hacker News Reader

Plaintext is not a great format for (system) logs

utcc.utoronto.ca

12 pointsthunderbong14 comments

When your "compressed database" is both larger and slower to query than the plaintext, has an unstable on disk format, and isn't even crash safe like journald's database I have to assume it's a sick joke and not a serious upgrade from plaintext logs on a (compressing) file system.

You can make it all some kind of json object and use plenty of tools to parse.

I think text is really awesome though. You can read it with any editor and there are a zillion easy to learn Linux tools like grep, cut, awk...etc for interactively creating a oneliner in a short amount of time that you can use to get what you want out of the text. I used to have a bunch of those when I was in charge of some applications on a Linux server. Text is very universal and I'm not just talking about log files.

Missed opportunity: it would really be a different conversation if, around 1980 or so, syslog(3) became a standard and was adopted at that time by everyone. Logs really would be in much the same format and you could use simple text tools on them.

After you agreed on the standard parts timestamp, facility, and level, you'd still need some standard structure for the variant message parts though, which is where json is good: it's arbitrarily deep.

https://www.man7.org/linux/man-pages/man3/syslog.3.html

Absolutely nothing wrong with binary logs as long as the system provides a utility to get the data out of that format.

For that matter, journalctl is terrible, cryptic, and difficult to use.

The title should be "Plaintext is not a great format for (system) logs for people have no idea about regexp" :)

In my opinion, plain text wins for everything, very portable and people can read with very little knowledge.

plaintext and grep were find for your grandpappy and they're fine for you.

Au contraire, plain text is just about the only format for logs that works. Having to use some obscure tool to query logs just isn't worth it, especially not in 20 years time, when you still need to analyze said logs but the tool doesn't even run on any contemporary systems anymore and the documentation about the binary format has been irretrievably lost.

And yeah, sure, you want to use structured logging. So, in addition to the greppable log message, include a Magic Separator Character (say, \t) that you treat as 'end-of-line' in human-oriented processing, and have your key-value-pair structured data following that for automated tooling to have its way with. Or, be really creative with [key=value] blocks that are both human- and machine-readable.

Some of my most painful logging experiences were having to extract application logs from binary blobs created by a proprietary Windows tool (no, not the main Windows event log: that's bad, but at least documented). Even the most recent versions of the official viewer just crashed, the vendor was not interested in fixing that (since we were not a customer, just a third party in need of log data, but even an offer of a modest payment was met with indifference...), and the actual format turned out to be byzantine beyond words.

So, yeah, give me plain text all day, every day...

On the other hand, multiple competing binary log formats make it harder to process the metadata than plaintext.

text is great for awk, grep, etc

perhaps some llm-powered awk command generator would be useful?

if only syslog() supported something like libxo...

tldr: the author confuses plain text (aka, storing in ascii files) with storing in some binary format (whatever it is). Said confusion makes the whole post worthless, in my opinion.

Where is this confusion made? The author compares syslog-style logging (plain text, with some tools grafting on ad-hoc metadata embedding formats) with systemd-style journal logging (binary with special tools to read them).

A log entry is composed of many things: some event nature, a user, who sent the entry, when it happened etc

  The core problem is that log messages themselves almost invariably come with additional metadata, often fairly rich metadata, but if you store things in plain text it's difficult to handle that metadata

So this is wrong: there are few "metadata" with logs : the "metadata" are part of the log, are stored with the log, they are the actual data for which we generated the log.

It has nothing to do with "plain text": structured-logging is a thing, and it is plain text.

And writing random english sentences to a mysql database is also a thing, and it is not plain text.

And you can move from one to another, just as you can compress a logfile (which is then no longer plain text) etc.

So the plain text vs binary is a format issue, and the logs' content is another, unrelated topic

So I do not know what's the point here, except saying "journald is awesome", which is just an opinion

This is fair. However, I think the situation the OP is complaining about is not one application that does structured logging (where I agree with you that even if the data is "plain text" in some sense, it's not terribly productive to think of it as plain text), but instead about a system log that many unrelated applications write to. Each of those applications has their own way of structuring their logs: there is no standard. Hence you end up with something that is technically plain text, but also really nothing more because there is no consistent structure. So it's really just plain text.

Reply on news.ycombinator.com