Toolong: Terminal application to view, tail, merge, and search log files
github.com
github.com
My fav option in this class of apps: https://lnav.org/ It lets you use journalctl with pipes as requested here: https://github.com/Textualize/toolong/issues/4
- System tool niche that I find interesting so it's nice to see a fairly complete and yet relatively small project
- Textual seems to be their largest dependency, but other than that the project seems self-contained and relies on the standard library
- With some exceptions, the majority of methods are short and easy to parse
- Typing support :)
But after a second glance it looks very well written compared to many other python projects, which sometimes read like a 5000 line bash script.
And I can't argue against your arguments, especially using "minimal" dependencies and using typing.
Typing often helps for autocompletion and understanding what a variable/function "means", which makes it [1] easier to start hacking on it.
[0] not necessarily bad, just wasn't what I would expect to be a small reference project
[1] not always, sometimes types can be too verbose and start messing with your brain ;)
I appreciate the work you are doing by dogfooding textual and how consistent you (or your team, I didn't know there was more than one person behind it) have been.
I just wanted to say thanks so yeah, Thanks!
Its relatively easy to add new formats in code, but there isn't yet a way of configuring that. In the future I might add a config file to make that easy.
Or some other similar reminder -- perhaps just a list of things you're going to install when you need them, so you don't install them until you have a use.
(I have this problem too)
Usually I don’t even need the reminder, just writing it down is reminder enough.
alias tail='echo "use toolong instead"
You might want to open an issue on the repo.
I had to replace line 4 of a 200 GB SQL dump, it took a substantial amount of compute time to perform a find / replace with sed and it also required having over double the disk space since sed creates a temp file before it writes out the new file.
Using a hex editor could have worked but it seemed too risky because data integrity was really important.
I don't remember exactly how long it took but I remember it being something like 30 minutes of purely waiting for the find / replace to finish on a 4 CPU core / 8 GB of memory machine. Memory wasn't an issue fortunately.
The syntax escapes me at the moment, but I'm quite sure I've used sed (or maybe awk?) in the past to do exactly that
I used "quite some time" because this happened months ago and I don't recall the exact time on the delete. I don't think deleting a line was any faster than doing a find / replace on 1 line. If sed is reading and writing the file in both cases I'd expect both to have the same performance.
For example if you're doing a SQL dump -> import, technically you could have downtime during this process to eliminate any chance of data loss. Having to wait ~30min for a command to finish is painful.
I'm not saying to go off and make this product to sell but if such a tool existed and you positioned it at $39 or whatever, if it could eliminate half an hour of downtime for a business it pays for itself. Especially if the alternative is to muck around with hex editing the file.
If 100,000 people have this problem and 3,000 of them would pay for it that's $117,000. If you spent 6 months making a super polished tool that made it easy to know exactly what's being edit (or deleting lines, etc.) that's pretty appealing. Even if the sales were half that's still solid but it could also be 5x too, who knows. Maybe you can finish such an app in 3 months instead of 6, etc.. It's also an app that mostly feels like it could reach a "done" state with little maintenance since you're editing text files which is a well known topic.
head -n4 sqldump > sqldump2
tail -n+5 sqldump >> sqldump2
This still requires double the disk space, but at least shouldn't take 30 minutes.
YMMV, but I just reproduced this on my machine, and the tail command took 3 minutes. Maybe you are limited by disk IO?
What was the sed command you used?
EDIT: sed was even a bit faster with:
sed -i '4s/^.*$/your line four/' sqldump
It makes sense for sed to write a temp file even with `-i` because what happens if your power goes out mid-way through the command without the temp file? You'd have data loss. To combat that sed will write a temp file out and when the operation is completed the last step is to move the temp file to the original file to overwrite it. You might not notice this unless you monitor the directory for changes while sed runs. You can spam `ls -la` as sed runs on a decently sized file to see the temp file.
Hold it like so:
seek=$bytes-offset-until-line-4 conv=notrunc,sync
Didn’t find this in the help/GitHub readme…