Atuin – Magical shell history
atuin.sh
atuin.sh
But above all, it's a solution to two real problems I had.
I work in several terminals. Sometimes tabs or Windows in my emulator, sometimes screen or tmux. And all those sessions would overwrite eachothers history. I lost many actual important history entries that way. And I (almost) ran many wrong commands, expecting another one to be my last entry. Arrow-up enter. Woops.
Atuin solved this.
I had my history as large as possible. Ctrl-r is my friend to search for "that old thing I did to pgsql piping something from zcat a year ago". But it's ever slower, zsh and bash history clearly not designed to handle (tens of) thousands of entries.
Atuin solved this.
Just checked yesterday* and have well over 100k lines in bash history and it loads / searches instantly in skim.
* Was trying to check out mcfly vs atuin for the third time and again didn't understand why the extra complexity was worthwhile.
# append rather than overwrite
shopt -s histappend
# attempts to save all lines of a multiple-line command in the same history entry
shopt -s cmdhist
# with cmdhist, saved with embedded newlines rather than semicolon separators
shopt -s lithist
HISTCONTROL=ignoreboth
HISTSIZE=10000
HISTFILESIZE=20000
HISTTIMEFORMAT="%y/%m/%d %T "
HISTIGNORE="history:ls:l:pwd:exit:"
if [[ ${BASH_VERSION:0:1} -gt 5 || ${BASH_VERSION:0:1} -ge 5 && ${BASH_VERSION:2:1} -ge 1 ]]; then
PROMPT_COMMAND=("history -a" "history -c" "history -r")
else
PROMPT_COMMAND="history -a; history -c; history -r"
fi
- And about the 2nd issue, you should use fzf or skim (probably faster) to replace the ctrl-r binding. fzf includes such a script that you just need to call from .bashrc. # Enable key bindings for fzf
if command -v fzf 1> /dev/null && [[ -f "/usr/share/doc/fzf/examples/key-bindings.bash" ]]; then
source /usr/share/doc/fzf/examples/key-bindings.bash
fiI've been using atuin happily for a few years now and it blows bash history out of the water.
I have far more confidence in atuin handling race-conditions and upserts than in my bash-spagetti not doing that.
Also, fzf, has a better append-history, but even there I managed to get it into some race-condition now and again. I presume it has to do with when the append happens: after the command ran and finished vs when I hit enter? Again, I don't know the details, and frankly, don't really want to know them either, if I can just lean on a tool by people who truly understand the problem.
Good, but even better (IMHO) is to write one history file for each terminal window, shell and level as in
export HISTFILE=$HOME/.history/${TTY##*/}.$SHLVL
And then I tell zsh to add timestamps setopt APPENDHISTORY # don't overwrite HISTFILE
setopt EXTENDEDHISTORY # add timestamps to HISTFILE
setopt HISTIGNORESPACE # aka setopt -g
set HIST_FIND_NO_DUPS # don't show duplicates on ctrl-RNow I can grep my history and, based on other work notes I take, can even limit my search by date/time if needed.
I use it every single day that I’m at a computer. It's very easy to learn (2 or 3 essential hotkeys), and makes finding old shell commands a breeze. I was able to self-host the sync server in well under an hour, start to finish.
(@ellie - if you see this thread, thanks for all the elbow grease you put in! You’ve built something really special.)
I'm not asking sarcastically, either. Is this something awesome I should be using? What about it is going to blow my mind?
I guess I’m looking for more of a personal experience anecdote, where someone could explain how its other features made it worth signing up for another sync service and learning a new thing. It may be the best thing since sliced bread but that page doesn’t say why I should try it.
- this matters because it's capturing not just what commands were run, but metadata about the commands as well. Of particular interest to me is that ot captures the start and end time for every history entry so you can get timing data even if you forget to use `time`
- it provides much more fine-grained control over what makes it into the database than shells provide for controlling what makes it into their built-in history. For example, it has built-in support for filtering out common kinds of secrets/tokens/etc.
It's also in active development with a friendly and helpful creator/project lead, an active community, and an ever growing feature set.
I'd recommend giving it a try; as far as I'm concerned, it's best-in-class for managing shell history
That’s potentially an incredible useful feature to integrate with all that metadata! Very very interesting.
- Filter by commands run within the current shell session
- Filter by commands run within the current working directory
- Filter by commands run across hosts (as opposed to filtering commands run on your local machine)
- Filter by commands run within the current shell session
- All of the above searching functionality, with nice fuzzy finding support, time stamps, etc.
Before atuin I used zsh's builtin history, with ctrl+r rebound to present that builtin history through the `fzf` fuzzy finding tool, and zsh configured to share history across shells. The deficiencies I found: I couldn't optionally filter by commands _only_ run in the current shell, I couldn't filter by commands run in the current directory (useful for quickly finding commands I often need to re-run for a given project), and I can't search for commands run across hosts.
If you don't find yourself valuing these things, you may find that you have little to gain from using atuin.
I use it mainly because a) it stores data about command history in a proper database that's easy to query (what century is this? How do other tools justify dumping what should be structured information into some god-awful mess of semi-structured text files?) and b) it makes a clean separation between SESSION, DIRECTORY and GLOBAL scopes which means you can explicitly use a recency or a locality filter (I don't want irrelevant commands cluttering my history).
Even without sync, atuin is a gem.
function hist() {
print -z $( ([ -n "$ZSH_NAME" ] && fc -l 1 || history) | fzf | sed -E 's/ *[0-9]*\*? *//' | sed -E 's/\\/\\\\/g')
}I think you mean "I have a one-liner zsh function that does a tiny subset". It's lacking:
- The ability to filter by current directory
- The ability to filter by current shell session, instead of across all shell sessions (that's assuming you use zsh's shared_history; if not, then the opposite is true: you can only search within the current shell session. See https://zsh.sourceforge.io/Doc/Release/Options.html#index-SH...)
- Search history across hosts
Until recently, I used zsh+fzf, with the default ctrl+r binding replacement provided by fzf. It's been great, but it has lacked functionality that I've wanted for a while now. Atuin fills in these gaps for me.
Atuin stores _everything_ it can about each command run, what you see when you press C-r is only a tiny subset. And even it gives you the duration and success/failure information immediately.
If you want to, try pressing C-r, select a command from history and press C-o. Normal shell history doesn't store any of that.
First, a bit more about my zsh+fzf use: I use `share_history` so that my history is available across shells, allowing me to quickly find and re-run commands. (https://zsh.sourceforge.io/Doc/Release/Options.html#index-SH...)
Sometimes I want to switch back to some terminal tab/window and find a command that I nkow I ran in that session, but now it's flooded with 1000+ commands that I've entered elsewhere, making it difficult to find what I'm looking for. So the shared history is both useful and a hindrance, depending on the scenario.
Compare with atuin: I can filter by commands entered specifically within the current shell, or across all shell sessions, and possibly even across other hosts (when using atuin's sync functionality -- haven't tried that yet).
Given the metadata that atuin collects (current working directory, start time, exit status, duration, etc) I'm sure there are some other clever things that can be done with my shell history now, vs the the "line per entry, with optional start timestamp" approach used by zsh's builtin history functionality. Actually, more on that first point: you can also filter by commands entered within the current directory.
It syncs across devices. I’ve got a number of raspberry pi devices I use and I never remember what command I used. Also preserves history when I inevitably need to replace the SD card.
Atuin also preserves information about the exit code. So you can filter on commands that worked. Which is great.
And, you can still use fzf to search your history if you want. I’ve got ctrl-r bound to search Atuin with fzf and ctrl-t to use native Atuin search.
If you’ve got an extensive history already, Atuin easily imports it. So I didn’t miss anything.
And because I wrote an ansible playbook to install it everywhere, including provisioning my self-hosted Atuin server, it just works.
Biggest downside was compiling for Raspberry Pis - which may be running slightly different versions of Raspbian. The project doesn’t provide the arm7 binaries. Eventually I figured out how to do fully static compilation with rust and that solved my problems.
Atuin replaces your existing shell history with a SQLite database - https://news.ycombinator.com/item?id=38936102 - Jan 2024 (10 comments)
I quit my job to work full time on my open source project - https://news.ycombinator.com/item?id=38935205 - Jan 2024 (176 comments)
Atuin replaces your existing shell history with a SQLite database - https://news.ycombinator.com/item?id=35839470 - May 2023 (193 comments)
Atuin replaces your existing shell history with a SQLite database - https://news.ycombinator.com/item?id=35688117 - April 2023 (1 comment)
Atuin for zsh shell history in SQLite - https://news.ycombinator.com/item?id=31177943 - April 2022 (1 comment)
Show HN: Atuin, improved shell history with multi-machine sync - https://news.ycombinator.com/item?id=27079862 - May 2021 (9 comments)
I also find that I prefer the default Ctrl-R to start with history from the current directory instead of the global history, but that is also changeable. So I’m pretty happy.
[0]: https://docs.atuin.sh/configuration/key-binding/#disable-up-...
Adding it to Atuin will take a bit of work to keep it from being too brittle, so it hasn't yet been done.
atuin init fish --disable-up-arrow | source
in my fish.config. C-r with Atuin is great. It taking up the whole screen when I want to run the previous command again? Not grat."A broken workflow is a broken workflow" is just a tautology. I guess you're implying "any kind of broken workflow is bad" but yeah, I don't agree with that especially with regards to free tools. The project literally has weekly commits, devs are open and engaged and... patches are welcome
I just wanted to respond to this a bit more. My moving away from a project is my choice. Just because it has active devs and regular commits doesn’t mean that I’m in any way obligated to continue to use it, or contribute to it. It doesn’t fit my workflow, and the issues are longstanding and glaring. Might they be fixed in the future? Sure. Am I interested in sending patches in or tracking that effort? Actually it really doesn’t even matter, a broken workflow is a broken workflow, even when the tool is free. It doesn’t work for me as is and I will find something else if I want.
This seems to be some sort of defensive response as if I was attacking the project. On the contrary, and like I said before, I generally like it, but the issues I have, which are to be both glaring and longstanding, means that it doesn’t fit my workflow and I’m considering moving. Not all tools need to suit everyone’s workflows, and “patches are welcome” is very beside the point.
Feel free to open a feature request before I do.
https://news.ycombinator.com/item?id=35839470
https://news.ycombinator.com/item?id=38936102
https://news.ycombinator.com/item?id=27079862
https://news.ycombinator.com/item?id=32491840
https://news.ycombinator.com/item?id=31177943
https://news.ycombinator.com/item?id=35688117
https://news.ycombinator.com/item?id=38969077
https://news.ycombinator.com/item?id=37127909
https://news.ycombinator.com/item?id=35876290
https://news.ycombinator.com/item?id=34972384
https://news.ycombinator.com/item?id=33786580
Also, even the possibility that the software would send this to the outside would make this impossible to use at my company, and I don't think we are overly strict in that sense here.
Why not? I use it without syncing just for the search functionality which is really useful to me and saves me quite a bit of time. I'm sure the syncing is useful but I don't care about my shell commands enough to want them synced across all the machines I use.
> If you would like to sync your shell history, registration is required. Otherwise, you can use Atuin locally as a fully-offline enhanced history search tool
It does not matter if you think that the website does a bad job of explaining that fact.
Rest assured, that your sentiment, that no employee should be using autin (or any locally installed 3rd party software, really) before a proper audit of the code has been done is understood just by your first comment. A valid opinion anyone can hold.
If the author of Atuin maybe sees this: While this not (yet?) a commercial project, it is highly problematic to advertise your product like this. You cannot just put the logos of companies on your front page without permission, even with that carefully worded caveat in front. At the very least, this can lead to a C&D.
I work/ed for some of the companies on the list and endorse his web page. The turtle logo is cute, seems fine to me.
When I start a new shell, it starts a new history with a generated meaningful name. That is good for one-off things and experiments. History is still always preserved. Nothing gets lost.
Most of my work is done in (usually long-lived) manually named sessions, that write their history to a file named like the session. I can restart old sessions any time.
Everything is in plain text files in ~/.history, which has pros and cons. One advantage is that I can archive older history files very easily. I often ripgrep through my whole history as well, though I occasionally long for it being in a proper database and content being more structured.
I can share my history sessions and use them from multiple shells simultaneously, but practically never do that, so it is not an important use case for me.
Would Atuin bring any benefit for me?
It wasn't until I switched that I realized how poor bash and zsh are in comparison, even with fzf
10 min install, never looked back
Not to mention the safety. Never could figure it out, but once every 15 months or so, zsh would disappear my history. Certainly a mistake on my end, but still too easy to blow away history
A few days ago I actually made my history file append-only to prevent truncation.
I've found these are usually a bit of an afterthought, involving a bunch of service dependencies and an arcane config process.
Postgres 14 and a few tweaks to a TOML file and you're running.
Not wild about the password-only Postgres connection though.
>> A valid PostgreSQL URI, for saving history (default: false)
Looks like you are free to specify socket or ssl if you prefer?
I was recently shocked to learn that the postgres standard "sslmode=require" only check host name vs cert - not if the cert is trusted (ie meaningless).
https://www.postgresql.org/docs/current/libpq-ssl.html#LIBQ-...
There's been a recent update (not sure which version, I don't have my Linux box on hand to check) where it now started logging some timeout waiting for a lock on the database.
I don't type commands in multiple shells at the same time, so not sure what's locking the db. I also don't use remote sync (explicitly disabled in the config).
- It turns out I basically never want fuzzy search through my command history, and certainly not by default. I gave it a try for a couple weeks but it was very frustrating to be searching for a particular command, type in the exact prefix, and have the thing I was looking for hidden among hundreds of irrelevant entries. Solution: search_mode = "fulltext" in Atuin's config.toml
- Having a full screen pop-up appear whenever I hit up was really jarring, especially since I have a habit of hitting up a few times when I'm at the command line thinking of what I need to do next, to sort of refresh my memory on what I was just doing; the popup very effectively destroyed that chain of thought. Solution: eval "$(atuin init bash --disable-up-arrow)" in .bashrc
These are pretty minor issues and it's possible my preferences are just different from most!
Atuin now works really nicely for me. My only outstanding issues are:
- Under mosh the UI ends up corrupting the screen; apparently this is really more of a mosh bug (no alternate screen support) and you can work around it by having tmux/screen running: https://github.com/atuinsh/atuin/issues/1324
- I still don't have a great model in my head of how sync works and find myself occasionally force-syncing across a few systems until I convince myself everything is in the same state.
- It would be nice to have some kind of settings sync so I don't have to make the config changes mentioned above on 10 different systems. Surprisingly I don't see a feature request for this yet so maybe I'll go open one...
Anyway I don't want these issues to stop people from trying Atuin – it's a really nice piece of software. I almost never make changes to the default environment so I consider it a testament to how useful it is that I've added it to all the systems I use regularly!
Also the stats are a neat add-on.
Overall would recommend to almost any developer.
We currently install `fzf` with the proper keybindings, so Ctrl-R is already quite nice. But we also already include one interactive "pick your $EDITOR" moment - there's no reason we couldn't provide a "pick your $HISTORIAN" with atuin vs fzf as well. It would be fun and might turn people onto new tools, which is a big aim of this project.
I wonder however how people can spend a lot of time in the shell and have no sync process at all in their shell history.
Atuin works best in bash when using ble.sh.
However, whenever I try to use it with ble.sh, it doesn't bind to the up key like it does with plain Bash.But it should still work
But since I use different flavors of UNIX and Linux some of the commands are different.
I think it could get confusing.
Obviously you could just not synch between platofrms
The reason it saves every command is so that it can produce statistics, for example exit code distribution and runs per day etc.
The runs/code can be stored as list to get the distribution, though for typo I don't want any distribution, I just want them gone, and for identical commands with different comments I'd want a single run stat
These are the cleaning type of things that would be closer to magical
It feels like you are just trying to find issues without having tried using it for real...
> closed and fixed ... 'list' command is supposed to list all
This is NOT a fix, but a confirmation that this design flaw is unlikely to get fixed. I don't need fzf to be flooded with useless info either.
How would using it for real help here?
Mainly a zsh user & I've looked up & down & all around, but I cannot find a way to keep track of what I actually typed, versus what got ran! I really want to better be able to identify patterns, look at what I was typing. Show me !-1$, the last word of the last command!
I need this to learn & improve my expansion capabilities better. I need this to see how my history evolved, to backtrack & see where else any given line was pointing at. Shell history feels devoid of the most important context I craft; I summon birds from hats and my shell history just says: there was a bird here. Hiss boo.
[1] - https://starship.rs/