Hacker News Reader

Comment by mgaunard | Hacker News Reader

Most logging libraries are bad because they're not designed to account for the requirements of a real-time system.

What you need:

- an asynchronous design where both the formatting and the writing to disk can happen on another thread.

- a lockfree buffer using pre-allocated memory. In particular every log record might be of different size.

- the ability to control where/when to drain that buffer, perform formatting and write data to disk (no good library should impose a threading model, needs to be able to cooperatively run with other tasks on one thread)

- an extensible and fast binary serialization/deserialization scheme -- easiest is to rely on simple copy construction as a default.

Most libraries focus on silly things like formatting and filtering which are all trivial problems.

View on HN

> Most libraries focus on silly things like formatting and filtering which are all trivial problems.

I wanted to print structured data on stdout, without any fuss. All the other libraries that try to tackle the non trivial problems you mention, feel like a bazooka aiming at the ant that is my use case.

FWIW, in the small desktop app I'm working on now, this library would fit my use case perfectly. Certainly the features mentioned above are powerful, but I think you're right in your belief that plenty of projects don't necessitate such complexity.

They are bad for real time, but not all libraries need to be designed for real time or low latency systems.

Right. We did exactly that, for the reasons you wrote, starting from [0] and making it pre-allocated and async. Formatting is as late as possible, even after we store the static and dynamic parts separately in a db. Very versatile (can log to console, json, web frontend, pandas,...) and space and resource efficient.

[0] https://youtu.be/FyJI4Z6jD4w?feature=shared

I have often written my own logging implementations for similar reasons. I think it is pretty difficult to turn the set of features into a generic library because it necessarily is opinionated about multiple aspects of the software architecture that uses it.

Not all formatting is amenable to being pushed to another thread without introducing undesirable concurrency concerns.

That's why you need an extensible serialization system: to handle data that isn't copyable or whose copies would require memory allocation.

Which my library propose with the very simple concept `as_string_trait` that requires the presence of a method `std::string as_string() const`.

It's good enough for 100% of my use cases.

well clearly that doesn't work, you can't pass it a buffer to write into.

This necessary allocates, and I guess it formats on the hot path too.

Which is good for 100% of my use cases, since I don't deal with real time systems, or performance intensive applications, or memory/allocation constrained environments.

No need to use the bazooka to kill an ant.

I would agree with all of this, _except_ maybe the first bullet. Async can be fine so long as there's a mechanism for flushing all buffers before crashing. Otherwise, there's a high risk that you're gonna miss logging an important event that helps pinpoint why the crash occurred

Those requirements can be met by writing your log data to a pre-allocated memory buffer and flushing the buffer yourself.

Reply on news.ycombinator.com