dimonomid

I do find the LUO support really interesting, especially in the context of Orphaned VMs.

Yes, LLM is a tool. And CEOs of the tech companies are retaining developers not because there is value in writing code by hand, it's because developers can use that tool effectively and responsibly: they can tell bad code from good code, they design the solution, let AI do the hard boring work of coding it out, then these developers check its output, test it, also they can be creative and e.g. ask LLM to review LLM's output, etc etc.

But the need to manually type stuff out arises very infrequently in my experience. It happens but it's an exception, not the rule.

Yeah, can't agree more, and my experience is pretty much the same as you describe.

Yeah a small amount of manually-written code still makes sense, when it's faster to write it than to tell LLM to write it.

In particular comments. I'm not a fan of the comment style LLM tends to write, so I often just write them manually.

And also coming up with the right architecture is also often manual job. Design how to implement things, and let LLM implement it.

Of course checking the LLM's output and testing it and iterating on it is factored in. And yeah that verification and testing and iteration where most of the time goes, and still it ends up being significantly faster than typing everything manually, and the end result quality is often higher I believe.

To avoid fighting with the LLM to get right, it's useful to first come up with the architecture manually, run it by the LLM first (without writing any code - just talking to it, asking it to find flaws in your architecture), and once both you and LLM are happy with the plan, fire it to execute. This step removes a lot of friction (but not all of it - later iterations are still almost always necessary)

Yeah if it's done as part of learning effort, then I def agree.

True, but having written enough (from many years of doing it full time), the returns from writing more are diminishing.

But yes I agree that as part of learning effort it's important to write, too.

It's very important to understand what's happening, sure. Blindly shipping AI-produced code without understanding is just irresponsible. But how does writing code helps with that? Reading helps a lot, but writing just slows things down in my experience so far.

I'd love to see open source win, for the reasons the article talks about and others, but so far I'm not convinced this is actually happening, unfortunately.

To me, literally writing code by hand instead of letting AI do it feels like digging a foundation pit with a shovel instead of letting an excavator do it. Sure it can be done but much slower and the benefits are questionable. But curious if you have reasons to believe otherwise?

Hey mcint, fyi both of these issues are addressed: the localhost one is addressed for real, and a Match issue is worked around: while it's still not properly implemented, at least it doesn't prevent Nerdlog from starting now. Just in case you wanted to give it another try.

Cheers.

Yeah, I agree. Gonna try and get it done on some weekend.

Thanks for trying it out, and for all the suggestions, very helpful!

Thanks for sharing! Good to know that nerdlog turns out to be helpful not only for devs (the original use case), but also for DevOps :)

Fyi, support for journalctl was added to master, in case you wanted to try it out. I didn't yet add automated tests with the mocked journalctl, but my manual tests show that it's working fine.

If a system doesn't have either `/var/log/messages` or `/var/log/syslog`, nerdlog will now resort to `journalctl` by default.

It can also be selected explicitly by specifying `journalctl` as the file, e.g. `myserver.com:22:journalctl`.

I see, interesting. If you don't mind me asking, is it a sysadmin kind of job? Just trying to understand the use case better.

Regardless, journalctl support is the single most requested feature, so yeah I'll at least try to make that happen; hopefully on the upcoming weekend if I'm lucky.

Yeah well the 2-files limitation is mentioned in some other place; but I agree this section could be rephrased to make it more clear too.

I only had the blood circulation issues from compression during sleeping, and also had no idea until today that the nerve damage like that is possible. I wonder how could it be prevented, if it can.

I seriously hope I won't have to deal with it, but thanks for expanding on it and the treatment.

Wish you a speedy recovery!

Damn I'm sorry to hear that. I'll see if I can get void linux installed somewhere to test it.

Good luck with whatever you're going through!

Fyi I've done a simple benchmark today, and journalctl is indeed a lot slower than simply reading log files. In case you're interested: https://github.com/dimonomid/nerdlog/issues/7#issuecomment-2...

I don't think there's anything wrong with this format, it looks good. The most important thing for nerdlog is that all components of the timestamp must be at the fixed offsets from the beginning of each line. So I believe it can be implemented, I can look at it on the weekend. Feel free to create an issue on Github (I won't be able to test it for real, so would need your help with that)

Fyi I was going to create a Github issue for the journald support, but apparently someone else filed it first: https://github.com/dimonomid/nerdlog/issues/7

Just posting it in case you want to subscribe to it. Looks like it's a popular demand indeed, so I'll at least poke it and see what kind of performance we can get out of it.

Sorry to hear you're having issues. I'll try to reproduce and fix the issue with the Match.

Not sure if that "Thanks" for releasing early is sarcastic, but regardless, I appreciate the feedback.

That's good news that we have /var/log/socklog/everything/current, but I'm also trying to figure the format. Is it like this? (sourced from chatgpt)

2025-04-21 12:34:56 myhostname myservice: Something happened

If so, then yeah it's totally doable to make this format supported.

Re: config.yaml, yeah I thought of that, but in the long term I rather wanted it to be nerdlogrc.lua, so a Lua script which nerdlog executes on startup. Similar to vim (or rather, more like neovim in this case since it's Lua). Certainly having config.yaml is easier to implement, but in the longer term it may make things more confusing if we also introduce the Lua scripting.

Thanks, appreciate that!

Re: the date/time format, I was thinking about implementing support for an option like timefmt, so you'd be able to do :set timefmt=2006-01-02T15:04:05Z07:00 , but postponed for now.

That's not hard to implement, however to make it persistent requires implementing some config / scriptability, which is a whole other thing and requires more thought.

Re: runit, I never tested it, but after looking around briefly, it sounds like there is no unified log file, and not even unified log format? I mean it's possible to make it work, treating every log file as a separate logstream, but I've no idea what these logs look like and whether supporting the formats would be easy.

Thanks. And no, as of today, there's no way to define a group like that. Might be a viable idea though.

However, before we go there, I want to double check that we're on the same page: this `log_files` field specifies only files _in the same logstream_; meaning, these files need to have consecutive logs. So for example, it can be ["/var/log/syslog", "/var/log/syslog.1"], or it can be ["/var/log/auth.log", "/var/log/auth.log.1"], but it can NOT be something like ["/var/log/syslog", "/var/log/auth.log"].

I first responded before your edit about ssh and localhost, so: yeah, as briefly mentioned in the article, as of today there's no shortcut even for localhost. I was debating whether I should implement this feature before open sourcing it, but I had to draw the line somewhere (I have TONS of ideas what could be implemented), and since reading local logs isn't the primary focus of nerdlog, I decided to skip it for now.

But yes the bypass for localhost can definitely be implemented.

Yeah you're right, agree with both of your points.

The `go get` one should be easy to solve though, and my bad for not thinking of it before, thanks. I'll look into it.

Thanks for the feedback!

Yeah it would be great, and I do want to support it, especially if the demand is popular. In fact, even if you ungzip them manually, as of today nerdlog doesn't support more than 2 files in a logstream, which needs to be fixed first.

Specifically about supporting gzipped logs though, the UX I'm thinking about is like this: if the requested time range goes beyond the earliest available ungzipped file, then warn the user that we'll have to ungzip the next file (that warning can be turned off in options though, but by default I don't want to just ungzip it silently, because it can consume a signficant amount of disk space). So if the user agrees, nerdlog ungzips it and places somewhere under tmp. It'll never delete it manually though, relying on the regular OS means of cleaning up /tmp, and will keep using it as long as it's available.

Does it make sense?

Thanks, and yeah getting distro packages would be dope. Hopefully, some day.

Not really, at least not yet, because nerdlog's focus is very different than that of lnav. There is a section about it in the article as well.

In fact nerdlog doesn't even support anything like -f (realtime following) yet. The idea to implement it did cross my mind, but I never really needed it in practice, so I figured I'd spend my time on something else. Might do it some day if the demand is popular, but still, nerdlog in general is not about just reading a continuous stream of logs; it's rather about being able to query arbitrary time periods from remote logs, and being very fast at that.