Hacker News Reader

Comment by lanza | Hacker News Reader

I love the idea of Atuin but it's just way too slow with large history files. I've synced my history on my own for the past decade and have like 170k lines and the history search the ctrl-r search just crawls.

I don't need most of the history, but there's 0 chance in hell I'm auditing that many lines to decide what I need and what I don't need.

View on HN

You don't need any of it.

History is useful enough to exist as a feature, I up-arrow routinely, but it doesn't actually matter when it doesn't exist.

I find the idea of going out of your way to preserve and migrate years of shell history and make it searchable in a db about like:

You have a problem that water is flooding your kitchen floor. Normally you deal with a spill with a mop or towels. There is now too much water and so you decide that your normal towels aren't good enough and so you get more & better towels, or even put a sump pump in the corner to keep pumping all this water away.

I've written a lot of complicated pipelines and awk and sed etc, but they were either one-offs that are of hardly any value later, or I made a script, and the few things that are neither of those, are so few they automatically don't matter because they are few.

It's not illegal or immoral, just goofy.

I use those ! Bash history features all the time. I.e. !?some_test to just rerun a test case I ran several months ago. I don’t need to sync histories between PCs (they are different enough) but history is important.

History can also contain potentially sensitive things like hostnames of non-public systems, usernames, filenames, URLs. I would not want that stuff to hang around indefinitely.

You can tell your shell to ignore commands started with a space

    HISTCONTROL=ignorespace

Or you can use HISTIGNORE to always ignore specific patterns.

You can also use

    HISTCONTROL=ignoreboth

to use both ignorespace and ignoredups

This is one thing that atuin announced to solve: ignore patterns, keys, passwords. From the announcement I got that it's going to be far more useful than HISTIGNORE. Both out of the box and in abilities.

> You don't need any of it. > History is useful enough to exist as a feature, I up-arrow routinely, but it doesn't actually matter when it doesn't exist.

That's absurdly naive to think the simplistic constraints of your own workflow is a general rule.

One can only speak meaningfully about their own experience and actual reasoning.

Anything else is bs imagination.

Had I not already allowed that "it's not illegal or immoral" or suggested the feature shouldn't even exist, you might have a point.

Maybe it does not matter for you. I have a big history file and with fzf that saves me a lot of time in remembering/rewriting commands.

Good tools change the way you work. Imagine if version control systems were notoriously unreliable and difficult to get working. You'd be saying "nobody needs their entire commit history, if you need a version create a source archive and back it up".

But version control systems do work, so we use them, we keep history and we tag releases and don't really need to bother with source archives any more.

Nobody is saying history is a substitution for documentation or an audit trail or anything else, but it is a useful tool if it works. Consider a case where you're exploring a new dataset you find online. You download it into a directory, run some commands to transform it, load it into a database etc. You don't know if this will ever be useful. But if it does turn out to be useful, you now have a log of everything you did to get there. Trying to document everything up front would be insane and you'd never get any exploratory work done.

You remind me of "hey, i just wrote a terminal emulator in JS".

I had a similar thought when I first looked at it, but then I thought about my browser history and URL bar. It is sort of a lot of work to open files to write scripts, keep them organized, and make them accessible just to make some commands simpler to run. I wrote https://github.com/ionrock/we for this very reason. I moved most args to env vars and made loading different env vars easily via files. Maybe the history is a better way to make these things reproducible and useful by avoiding the redirection necessary by scripts?

While I agree it may not work with everyone's workflow, maybe it could be a powerful change to folks workflow. I'm going to try it out and see for myself!

Does your history store frequency? Could prune those with only a single run until it's performant

remove duplicates, remove trivial commands like cat, less , more, cp etc

> but it's just way too slow with large history files

hey! what issues were you having here? slow to open or slow to search?

we have a whole bunch of people with way more than 170k lines, so that shouldn't be happening :/

I’m self hosting my ATUIN server (although it is offsite so about 50 ms away), and have right at 100,000 items as shown by the command atuin status I don’t have any speed or lag issues, it comes up instantly and searches instantly. Fwiw.

I Love the project (will donate now, I remember when the project wasn’t taking donations, and I suggested they should) Thanks

Edit: I should add: I have the client on about 15 to 20 different VM‘s, all with various OSes and versions. The server part I’m running w docker (I think, via the exact steps suggested in the docs). All Works great in my use case, and I do have some very long and complex commands that it’s storing

The delay between the physical keystroke and rendered text insertion + search filtering is uncomfortably high. Compared to FZF's instantaneous rendering of the same this is a UX downgrade.

Reply on news.ycombinator.com