tstack

I followed these instructions and the main advantage for us is that it can send notifications to phones and google/alexa devices. I ended up switching the software to sinric.pro, though.

> - regex101 support for quickly defining custom formats is just awesome. Versioning support is slightly broken however, probably because regex101 changed something, so there's no easy way to update the format once you've initially imported it.

There is a `pull` sub-command and it looks like it still works. Running the following will generate a patch file with the updated regex:

    lnav -m format <format-name> regex std regex101 pull

It creates a patch file since the original file might've been modified.

> - There are lots of different filtering capabilities, but there is no unified treatment of them. For example, `:hide-lines-before` and `:filter-out` are at their core the same type of operation: filtering. I should be able to pull up a list of all filters that are currently active and easily add new ones and toggle or delete existing ones.

Adding the time filters to the "Filters" panel sounds like a reasonable request. I've added https://github.com/tstack/lnav/issues/1275 to track.

> - I would expect to be able to create a new view of the data using SQL `SELECT`. A select statement is fundamentally about filtering out some rows (log lines), which feels like a filter, and selecting some particular columns (log fields) and hiding others. The latter point seems like it could be something that should be handled when https://github.com/tstack/lnav/issues/1274 is resolved.

There is the `:filter-expr` command (https://docs.lnav.org/en/v0.12.2/commands.html#filter-expr-e...), have you tried that?

> I do wish there was support for switching formats so I could switch between different "views" over the same data, maybe it will be possible someday :)

I created https://github.com/tstack/lnav/issues/1274 to remember this

> I can also edit them if needed.

That doesn't seem .. wise. lnav has support for filtering, bookmarking, and attaching tags/comments[1] to log messages so that editing the log file isn't required. The filters, bookmarks, tags, and comments are saved separately so they can be restored when the file(s) are reopened.

[1] - https://docs.lnav.org/en/v0.12.2/usage.html#taking-notes

Pretty much, yes, it's not for dealing with a bunch of servers.

There's some basic support for tailing files on remote x86 machines (https://lnav.org/2021/05/03/tailing-remote-files.html). But, again, just small scale stuff.

I use it on my development machine and for going through logs attached to bugs. Those use cases aren't served by something like ELK/splunk/etc.

> That's funny, because that's been my experience with lnav! Not saying there's anything wrong with it though.

I tried to use the hotkeys from less/more/vim so that it would be somewhat familiar. I think people are frequently tripped up if files are not recognized as a log and just treated as text. Files treated as plain text are separated from log files, so it can be a bit confusing. Not entirely sure how to improve the experience there.

> lnav not recognizing log types I think it should have (Apache/Tomcat)

There are quite a few log formats builtin. But, since log output formats can be customized by admins, it's possible they deviate from the builtin ones and things won't "just work".

> not correctly loading custom log parsers

I've tried to improve error messages a bunch[1] and make it easier to trouble shoot configuration issues[2]. I'm sure more could be done, I just don't quite know what folks are tripping over without feedback.

> If you don't mind next time I try it I can give you more concrete info.

Feel free to file github issues or email support@lnav.org

[1] - https://lnav.org/2022/08/04/pretty-errors.html

[2] - https://lnav.org/2023/08/04/config-dump.html

LogViewPlus (https://www.logviewplus.com) is very similar to lnav and built for Windows.

I'm the author of lnav .. and not a very good writer, apologies.

I guess my main question would be, what are you expecting to get out of lnav? I use it primarily for merging log files together and just jumping around trying to understand what was happening. It has a bunch of other functionality, like using SQL for analysis, but that's not something I use regularly.

Really, a lot of the benefits of lnav are automatic, like uncompressing files, detecting log formats, tailing... So, if that's not something that comes up for you, it might just not be the tool for you.

I actually have this "not getting it" problem with VisiData/multitail. I start them up and they don't behave like I would expect when pressing hotkeys.

As others have kinda alluded to, it could be useful for testing TUI applications. I develop a logfile viewer for the terminal (https://lnav.org) and have a similar application[1] for testing, but it's a bit flaky. It produces/checks snapshots like [2]. I think the problems I run into are more around different versions of ncurses producing slightly different outputs.

[1] - https://github.com/tstack/lnav/blob/master/test/scripty.cc [2] - https://github.com/tstack/lnav/blob/master/test/tui-captures...

The Logfile Navigator (https://lnav.org) is a log file viewer/merger/tailer for the terminal. It has some advanced UX features, like showing previews of operations and displaying context sensitive help. For example, the preview for filtering out logs by regex is to highlight the lines that will be hidden in red. This can make crafting the right regex a bit easier since the preview updates as you type. lnav also has some simple bar charting abilities, so you can visualize the results of SQL queries made against the log messages.

Hah, I just started reading The Story of Diva and Flea[1] to my 5-year-old and "Flea" describes himself as a Flâneur. Still trying to figure out the right way to pronounce it...

[1] - https://en.wikipedia.org/wiki/The_Story_of_Diva_and_Flea

> As much as I love terminal, its just so limiting when it comes to browsing and filtering through more content.

What terminal-based tools are you thinking of when you say this? Just tail/grep/less?

> Looks like PRQL doesn't have a Go library so I guess they just really wanted something in Go?

There's some C bindings and the example in the README shows integration with Go:

https://github.com/PRQL/prql/tree/main/prqlc/bindings/prqlc-...

The Logfile Navigator (https://lnav.org) - A logfile viewer for the terminal. Have used it just about every work day for the past decade and a half.

A Closed and Common Orbit by Becky Chambers.

Yes, there were some roadblocks and that made me lose some steam. What's kinda funny is that I spent a lot of time trying to make pretty errors[1] in lnav as inspired by rustc, but had a hard time replicating that in the rust ecosystem. For example, I was using serde to parse lnav's JSON configuration files, but getting serde to report line numbers (and/or a JSON path) for errors was pretty difficult. I think serde was also missing a way to report multiple errors.

Otherwise, I had wanted to do some rearchitecting to do log parsing in background threads since it's largely sync in the C++ version. Doing that in rust was great since the borrow-checker guided me down a sane path.

[1] - https://lnav.org/2022/08/04/pretty-errors.html

I also wish for a rust replacement of lnav! (I'm the author)

I actually started working on one earlier this year and made some progress before getting distracted. I was using tui-rs, which then got forked to ratatui. I should check back in with it sometime soon.

Yep, exactly this. I wrote a short blog post[1] announcing the playground, but didn't go into much detail. All this stuff just runs on the free-tier of fly.io since it doesn't get much traffic.

[1] - https://lnav.org/2022/09/01/playground.html

> all so that the same binary runs on multiple operating systems, which isn’t actually very useful.

I like to mention my use case when this comes up: my log file viewer (https://lnav.org) uploads an agent to remote hosts in order to tail log files on that host[1]. While lnav itself is not built using cosmo, the agent is. So, it works on multiple OSs without having to compile and include multiple versions of the agent.

[1] - https://lnav.org/2021/05/03/tailing-remote-files.html

I’d be interested in what you found difficult to use lnav, if you have a minute.

> I also don't see something else I might want: a way to have a different "view" for certain log messages; maybe to switch between filtering/viewing particular ones, maybe to just have line-format be conditional based on the detected format.

Have a look at the following comment on an issue that might be similar to what you're thinking of:

https://github.com/tstack/lnav/issues/1065#issuecomment-1602...

> I guess I can sort of do this based on `module-field`? but I might want it lighter-weight/finer-grained than that.

Unfortunately, the "module-field" does not work for JSON logs at the moment. It's something I should really fix.

Ultimately, lnav has existed for almost two decades now and I use it every day. So, it's always seeing improvements. If you're having a problem with it, file an issue on github. I don't always get around quickly to fixing other folks feature requests / issues, but it tends to happen eventually.

Thanks.

> But if some log message uses an attribute you haven't anticipated in the format definition, there's no way to see it in the pretty-printed output

Properties in the log message that are not in the "line-format" are displayed below the message as key-value pairs. As an example, the bunyan[1] log format[2] has a few standard properties that are used in the "line-format" to form the main message. Then, as shown in this test[3] for this log[4], the remaining properties (even ones not mentioned in the format file) are shown underneath.

> or filter on it in the UI

Since they are part of the message as mentioned above, they can be filtered on.

> and no ability to see it in the SQLite virtual table.

The "log_raw_text" column in the table can be used to access the original log message from the file. So, you can use the JSON functions in SQLite to retrieve the value:

    ;SELECT log_raw_text ->> '$.repository' from bunyan

[1] - https://github.com/trentm/node-bunyan

[2] - https://github.com/tstack/lnav/blob/master/src/formats/bunya...

[3] - https://github.com/tstack/lnav/blob/master/test/expected/tes...

[4] - https://github.com/tstack/lnav/blob/master/test/logfile_buny...

There are certainly limits on what can be displayed and done interactively due to the low resolution. Especially, when it comes to charting data. Some visualizations are available within lnav, mostly bar charts and the like. What type of things are you trying to do that you think are not possible within a TUI?

lnav has support for JSON-lines, logfmt, as well as the Bro and W3C Extended Log File formats that are XSV and self-describing. The contents are also accessible through SQLite tables. Is there some gap here that you're thinking of?

The Logfile Navigator (https://lnav.org), a logfile viewer for the terminal. Started it in 2006 and have used it most every day since to view the logs of whatever software I was working on at the time.

I use an APE executable as an agent for communicating with remote hosts in the Logfile Navigator (https://lnav.org). While lnav itself is not built as an APE, the agent built into itself is. That agent is transferred to the remote when the user wants to read logs on that host. This way, there is no extra step to determine the type of OS or building in multiple versions of the executable. Here's a short blog post on the subject:

https://lnav.org/2021/05/03/tailing-remote-files.html

The tmux integration in iterm is also very nice for remote work if you haven’t tried it out.

Yes. I followed a tutorial for building a doorbell that uses an ESP32 to send a notification to our phones when the button is pushed. I haven’t touched it since installation and I’m always a bit amazed it still works. An RPi would’ve been overkill.

> Hmmm, it doesn't say in https://lnav.org/features#automatic-log-format-detection but i see that at least json (and xml) is mentioned under the pretty-printing header.

Yes, I should mention it on the features page. It's currently only mentioned in the main docs:

https://docs.lnav.org/en/latest/formats.html

The SSH demo was inspired by git.charm.sh