The most useful thing in bash | Hacker News Reader

The most useful thing in bash

coderwall.com

268 pointsbitsweet140 comments

Since this is ugly clickbait, here is the "content" of the linked site:

> Create ~/.inputrc and fill it with this:

    "\e[A": history-search-backward

    "\e[B": history-search-forward

    set show-all-if-ambiguous on

    set completion-ignore-case on

> This allows you to search through your history using the up and down arrows … i.e. type "cd /" and press the up arrow and you'll search through everything in your history that starts with "cd /".

clickbait? Are you suggesting that this would've been better as a Show HN post?

I have no problem viewing the site of someone trying to show me something useful.

The title could be factual instead of sensational.

I don't know, when I read the title the first thing my mind jumped to was CTRL-r command searching. I was thrilled to see something similar but better.

For highly subjective definitions of "better", that is.

well from my perspective:

1. I've definitely seen this before, perhaps on reddit

2. You press ctrl-r <type> to search back through history. Fucking around with up key mapping is retarded.

I totally agree with the title, this was the number one issue I had with bash default behavior

This is a killer feature of emacs and M-x shell for me.

Now my command history is searchable, just like any other file or buffer I have open.

I use the same idea for any repl, R,python,ruby,node,clojure. Bash is just another repl.

The major advantage of the emacs buffer method is that a lot of tasks I do on the computer span many commands. I can search for one command I remember executing 3 months ago, and then review the series of steps I took before and after, along with the output.

Typically finding a single command I've executed before doesn't help a whole lot. I need context.

M-x shell is one of the first things I just show people when I introduce them to Emacs. I had a databases class a few years ago where the CLI program we used to interact with the database was a basically an input loop where you entered commands. It had absolutely no concept of history though so if you messed up a query you had to retype it or use the clunky copy and paste feature of whatever terminal emulator you were using. Emacs changed everything though be cause I could just kill my last command and paste it into the new prompt. I have since done this with many other commands and I love it.

I'm an emacs lover myself, but for that use-case it seems like all you need is rlwrap: http://utopia.knoware.nl/~hlub/rlwrap/#rlwrap

I've recently been using shells inside Emacs exclusively and not even starting Terminal.app any more. I've got a couple of tips for anyone doing this, based on my (limited) experience:

1) Try eshell which allows you to use Emacs Lisp for shell scripting, as well as override some common programs to use Emacs instead (grep, man, etc). I'm someone who has never really bothered to learn shell scripting and am much more comfortable using Lisp, even Emacs Lisp which is sort of the crummiest of the popular Lisps (don't get me wrong I love Emacs) Obviously YMMV and if you know shell scripting this is a non-issue, but for me it comes down to being constantly surprised by the little bit of shell scripting I've had to do, and having Emacs Lisp work basically the way I expect (dynamic scoping, Lisp-2 and all)

2) Both eshell and M-x shell are not actually terminal emulators, so anything that requires a terminal emulator, like pagers, git log (ok this is a bad example since you should just be using magit, but anyway...) or some programs that update the terminal with their status act totally goofy and sometimes just break altogether. In this case, you want term-mode, which is a proper terminal emulator. Be sure to read up on the different modes as otherwise you'll get stuck inside that buffer ;) http://www.gnu.org/software/emacs/manual/html_node/emacs/Ter...

Hope this helps another Emacs novice... most of this I learned from here:

http://www.masteringemacs.org/articles/2010/11/01/running-sh...

C-r isn't much harder than up/down arrows (or C-p/C-n). Why is this such a big deal?

With C-r you need to first type C-r, then type your search key. With the OP's inputrc nice hack, you first type the begining of your command, and then up. To me it is very different.

With Ctrl-R it is the same. If you start typing a command, then type Ctrl-R it will use that as the first part of your search.

You sure? Not in the bash versions (3 & 4) I'm using... In anycase, I'll stick to C-R since I know it works universally. And I depend on C-R quite a bit.

yea it is worse

Ctrl+R lets you search old commands in a similar way. I would say it's more intuitive, too.

Not to mention that it doesn't require you to take you right hand off the home row.

And you get to keep using the up arrow to go to your previous command.

Not as cool as zsh's push-line, though: http://www.cs.elte.hu/zsh-manual/zsh_14.html

> And you get to keep using the up arrow to go to your previous command.

Ctrl+P :)

This is the truly great part. It's the same reason I use emacs.

use vim mode, then j,k.

Bind:

    Control-p: history-search-backward
    Control-n: history-search-forward

Then how do you go to the previous command without leaving the home row?

Still works if you haven't typed anything yet. If you have (and don't want it as your search):

    C-a C-k C-p

One history trick I use a lot that most people don't seem to know about is the !$ variable, which refers to the last argument of the previous command. It's basically a bash pronoun. !! is the entire previous command which is occasionally useful with find:

find . -name foo

vi $(!!)

and !N, where N is a number, is the Nth command in your history.

And !N:$ is the last argument of the N'th command in history. Even more useful, !vi:$ is the last argument of the last "vi" you did.

Woah, clearly I need to spend tomorrow re-reading the BASH manual.

I find alt-. to be more usable - I know for sure what it will do, as I see the effect of the expansion immediately.

sudo !!

Are you feeling lucky?

A usage pattern with !$ is:

  % mkdir foo
  % chmod g+s !$
  % ls -ld !$   # (make sure it's correct)
  % cd !$

Alternatively, you can add ':p' to the third line. This will print out the command rather than executing it. Additionally, it's also added to your bash history, so you can add it by pressing the up arrow.

  $ chmod 755 foo
  $ cd !$:p
  cd foo
  $ cd foo

There are some similar "tricks" here:

https://news.ycombinator.com/item?id=5337558

I have found "magic-space" more useful than ":p". This auto-expands history substitutions (like !$) when you press the space bar. You can bind it to the space key in tcsh or bash. It also works for !!, !vi, and other substitutions.

So, when you press ! $ <space>, it expands out to "foo" in place, so you can be sure that's what you wanted, and edit it in place.

I think :p predates magic-space. They're both decades old at this point though.

$_ is a variable for the same thing. The only difference winds up being the effect on the current line's result in your history: !$ is replaced while $_ stays $_ and so its meaning will change if you run the same command later, which can be what you want or not, depending.

You're right in this specific case, but there are other useful ! arguments, so it's worth learning both.

It's the "Event Designators" in the bash manpage for those who are interested.

Oh, I didn't mean "You should use $_ instead", just "$_ is also useful."

As I tried to indicate, you should ideally use whichever will make more sense in your history later. If the command you're typing should always be applied to that file, which happened to be the last argument of the previous command this time, then use !$; if the command you're typing should always be applied to the last argument of the previous command, and it just happened to be that file this time, use $_.

Agreed, one of my favorites. I do this so many times every day:

hg diff -b <filename>

hg commit -m "comments" !$

Thank you for pointing out that I can do per-file commits. I didn't know that before!

I use 'hg st' so much that I've aliased it as 'st'. It's just about muscle memory now.

I would do vi `!!` instead since the backticks are right next to the !'s already.

alt+. does the same thing as well, and is slightly easier to type.

    $ Make me a sandwich
    What? Make it yourself
    $ sudo !!
    Okay

Agreed - I recently devoted an entire blog post to making the most of !$, it's that useful to me:

http://geekblog.oneandoneis2.org/index.php/2013/04/03/protip...

I've always found interactive history UI to be difficult to work with beyond the most trivial cases. Instead, I'd always find myself typing "history | grep ..." over and over again. Eventually I wrote an improved history command with built-in filtering: http://curtisb.posthaven.com/a-better-history-command

It's funny how personal workflows can be. I do the history|grep thing too, but never felt the need to automate it further. It's a relatively rare thing (maybe once a day), used when I know I did something specific in the past but forget a detail or two.

Instead, I generally drop little shell scripts around my work area for specific tasks. Working on iterating a build/install to chase a specific bug? Stuff it into a little script. Writing a find command to munge a bunch of files with xargs or -exec or whatnot and realize I'm into the third line or editting? Stop and echo it into a file, then finish it in emacs.

The idea of doing all that junk and then relying on the bash history file (instead of, y'know, the actual filesystem) to store it for me seems weird.

Ctrl-x e opens the current line in your $EDITOR, smoother than echoing, especially when there are pipes involved. Save it out as a script from there if you like.

It's C-x C-e in my bash, fwiw. And I tried getting used to this but it breaks locality by dropping the new file into /tmp instead of $PWD. Quoting a line isn't really that hard, or you can always cut and paste with the mouse.

I use screens and session variables to manage context, so scripts don't need to be "around my work area" but rather in the appropriate ~/.context/$SESSION/bin directory (or ~/bin if sufficiently general).

Again, personal taste leads me to say "yuck" to this one. You don't think that the appropriate spot to do an ad-hoc build integration is the source directory? You'd rather put it somewhere that has to do with the window into which you're typing (which for obvious reasons won't survive a reboot or copy) and not a place that corresponds to what you're actually doing?

Ah, if something's particularly specific to a directory subtree, rather than a particular context/task, I'll drop it in the tree sometimes, for sure. But my contexts very much correspond to what I'm actually doing (and my prompt includes the session), and so ~/.context/$SESSION/bin is "a place that corresponds to what [I'm] actually doing".

Which is not to take away from your "it comes down to personal preference" by any means - just trying to make mine a bit clearer.

How do you persist these scripts across sessions?

I'm not sure just what you mean, but to try to answer: $SESSION contains a meaningful context name, not a nonce string. Stuff in ~/.contexts/$SESSION sticks around unless I manually remove it (I also store context specific history and config files there). If I need to access a script from another context, I can just give the full path - winds up working a little bit like namespaces.

Acme has an idea of guide files in each directory which store commands you commonly use in that directory. Acme maps a mouse button to "send this text to sh", so you can edit a command in the guide file, highlight it with a sweep of the mouse while holding down the left button, then release the left button and middle-click to run it in the context of the current directory. Over the years this mechanism has greatly improved my retention and usage of interesting and powerful shell commands, and decreased to near zero the amount of time it takes to construct the command and the number of errors I make while doing so.

It's funny how personal workflows can be.

Very much so; much like the GP, I would "history | grep" and even had it aliased to 'hg' (which is one of the reasons I don't like Mercurial, just to start another flame war), but then I was reminded of C-r, which I should have known better (as an Emacs user (that's two flamewars . . . )), but oddly hadn't been using as I was just too lazy to really learn bash in depth.

One super-useful bash feature is anonymous named pipes, which allows the equivalent of multiple pipe "arguments" to a single command.

Instead of:

echo a b c > /tmp/foo

echo d e f > /tmp/bar

diff /tmp/foo /tmp/bar

Just do:

diff <(echo a b c) <(echo d e f)

That feature is actually named "Process Substitution".

"anonymous named pipes" is contradictory, since anonymous means that something has no name.

Yeah, poor wording on my part. They do have a name, but normally you don't care about it.

This feature is known as "process substitution":

http://tldp.org/LDP/abs/html/process-sub.html

I use pg up/pg dn for history searching and leave the arrows for simple up and down lines, like so (in zsh):

    [[ -n "${key[Up]}" ]] && bindkey "${key[Up]}" up-line-or-history
    [[ -n "${key[Down]}" ]] && bindkey "${key[Down]}" down-line-or-history
    [[ -n "${key[PageUp]}" ]] && bindkey "${key[PageUp]}" history-search-backward
    [[ -n "${key[PageDown]}" ]] && bindkey "${key[PageDown]}" history-search-forward

That's why I use fish. Not only does it cycle like that by default, but it autocompletes entire commands, searches for the substring in the middle of words, looks better, is faster, etc etc.

I am really amazed more people on HN aren't using fish. Everyone should check it out: http://ridiculousfish.com/shell/

Does it run bash scripts or does it have its own scripting language?

Your script should have "#!/bin/bash" at the top, which tells the shell what interpreter you want it ran with. So, yes?

It is the kernel the one that interprets shebangs, not the shell:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...

It's unfortunately on the one hand too beholden to POSIX shell behavior, while still being too different from mainline Bourne derivatives. I've tried it a few times, and the cognitive load is too extreme for my 20+ yrs of POSIX brain-damage to work around.

I don't use fish for shell scripting, but I'm enjoying it as my primary environment after making these adjustments..

Status codes and argument lists:

    $status  # replaces $?
    $argv    # replaces $@

Aliases are just functions (configure your startup in ~/.config/fish/config.fish):

    # alias ls="ls -lrth"
    function lr; ls -lrth $argv; end

For-loops are familiar:

    # for n in one two three; do echo $n; done
    for n in one two three; echo $n; end

The lack of '!$' confounded my hard-wired fingers until I settled on Alt-u to avoid up-arrows (see /usr/share/fish/functions/fish_default_key_bindings.fish):

    bind \eu history-token-search-backward  # replaces !$
    bind \ei history-token-search-forward   # bonus fun

The remaining price for fish's pseudo-telepathic assistance is the occasional frustration with applications that rely on the value of a user's default shell (/etc/passwd).

You can also get the same functionality with ZSH using https://github.com/zsh-users/zsh-history-substring-search

I tried oh-my-zsh for that, but it was very slow to start up. I went back to fish when I realized that I had basically set up zsh to act exactly like fish.

I tried fish and went back to zsh when I realized that I was basically setting it up to be exactly like zsh. :P

Thanks for the tip. I just installed it, and it seems really nice. The autocompletion of flags is pretty amazing.

I've been using Fish on my Mac for a while as well as some VPS' I have. At work I just installed it on our dev server.

It has a build in git status script you can include in your prompt that's awesome. Here's my fish prompt (note __fish_git_prompt):

    function fish_prompt
      set last_status $status

      set_color $fish_color_cwd
      printf '%s' (prompt_pwd)
      set_color normal

      printf '%s ' (__fish_git_prompt)

      set_color normal
    end

Sample:

   ~/G/Calendar (branch_name * ↩ =)

For bash/zsh users: https://github.com/git/git/blob/master/contrib/completion/gi...

Or there's always "set -o vi" then you can search through your history with /foo and n to find the next match.

What does this give you over the default Ctrl-r?

my thoughts exactly

Ctrl-R finds commands that contain the string anywhere. The method the OP posted finds commands that begin with the string. This is an important difference, and if you give it a try, you'll likely agree the latter is what you want most of the time.

ah ok. Thanks for the clarification :)

Thanks for the clarification - although depending upon how you work, this is not much of a benefit. For instance - if you type "/path/to/script.sh" you want to search for the name of the script - not the beginning of the path.

C-r in zsh takes regexp patterns (so you could do *^substr to match begins-only).. I hadn't noticed this wasn't the behavior in bash with readline.

Anyone know if that can be done in bash/readline?

whoops, errant asterisk that I can't edit out. unclosed attempt-to-italicize

In my workflow, I almost always want substring match as that allows me to input either the filename (object) or the command (verb).

It's not the same as Ctrl+R - it actually doesn't look for a match in the middle of the string, only from the beginning of it, so while it's better with going back and forth between the matches, it matches less data to begin with.

Awesome. I wrote another history tool called "list commands"

The idea is that: many commands are location-specific. When you type `lc`, it will only give you commands that were executed in that directory.

https://github.com/pconerly/lc-listcommands-bash

I would love to get some up/down arrow ability for `lc`. Right now I do a lot of:

$ lc | grep X

I also love the history-search-{forward,backward} shortcuts. I took over the Raspberry Pi SD card image generation process back when we moved to Wheezy for the 'official' images and put this shortcut in to the default install with the hopes of leaving it there if no-one complained. Sadly someone did, I caved in, and it is no more.

AutoJump is my co-pilot in these situations:

https://github.com/joelthelion/autojump

Something that gives this snippet much more weight.

export HISTSIZE=10000

Allows you to have a much bigger history.

You would also likely want to increase HISTFILESIZE as well. HISTFILESIZE sets the size of the history file, HISTSIZE is per-session

also

export HISTCONTROL=ignoreboth

so that repeated commands are only stored once. Handy for those, like me, who reflexively type commands like "ps aux" when thinking about what to do next.

My favorite history trick in bash is "cd -".

It works in tcsh, too.

you can also use the previous directory like this:

  cat ~-/sub/path/to/file

(that's tilde-minus-slash, which tilde-expands to the previous directory)

A funny story about "cd -". When I first started learning the command line during a programming course, they basically told us just enough to get by in order to complete the assignments. Somehow, "cd -" was mentioned but not "cd ..", so for a long time I used "cd -" as my only method for getting to the parent directory! (since that was usually where I had just come from). Reading about everyone discovering "cd -" much later in learning bash is very amusing, since I have been using it since day one (albeit in a totally incorrect way!)

And if you use ZSH you can go one step further with proper substring search.

https://github.com/zsh-users/zsh-history-substring-search

This doesn't work for me on Mac OS X Mountain Lion. I get:

    -bash: \e[A:: command not found
    -bash: \e[B:: command not found

when my shell starts, and it doesn't work.

just use oh-my-zsh https://github.com/robbyrussell/oh-my-zsh this feature is enable by default.

Thanks a lot. I never liked Ctrl+R. This is also the way Vim handles history...

Ctrl+R always seems natural to me, but maybe that's because I'm an emacs user.

I disagree.

1) If you work on multiple machines avoid heavy configuration and remapping of keys. It will frustrate you to no end. "Oh I forgot on this machine it is the up arrow on the other it is Ctrl+R" etc.

2) Up/Dn Arrows already do a useful thing -- they cycle commands in historical order. That is also useful in addition to the Ctrl+R history search.

I couldnt agree more. I had a knee jerk reaction to say that it would be someone who works in bash who would say something like this, because constantly sshing in to various boxes and having to constantly tweak your various . files is a fucking pain.

But after thinking it over, I definitely know those who are awesome sysadmins, and require every bit of the setup to match their preferences before they will deign to work in that system.

Use version control on your dotfiles.

If you do a backward history search with the prompt empty, it's the equivalent of cycling commands historically, isn't it?

You can still search chronologically by hitting up arrow without typing anything. Ctrl-R still works. The only thing that's frustrating for me is not having it on computers I haven't assimilated yet. ;)

Umm, .inputrc is readline not bash so technically this can be used in anything linked to readline?

Yes, including REPLs that use readline. Those that don't you can use rlwrap.

I get "\e[A" and "\e[B" command not found.

Otherwise it works. But ... ctrl U to clear the line, doesn't clear it completely. It only clears text from the start of the line to the prompt. What has been added from the history is left there one the line. I have to first move to the end of the line then type ctrl U to clear it.

My bash does this automatically and I do not have a ~/.inputrc file. I'm on Mac OS X 10.8.2 using the Terminal app (which is a bash command line interface). I assumed this was a default feature of bash but apparently not. If somebody can explain to me why I have this feature by default, I would be grateful.

Bash will up/down between past commands, but it's unfiltered. This lets you type the first few characters you know you're looking for to find it faster.

OHHHHHH. I get it now. Thank you for the explanation.

I use fasd: Its the best bash tool out there. Jump, search the previous history based on frequency of usage. It updates your search with its algorithm matching the most predicted one.

https://github.com/clvv/fasd

worth mentioning, nothing comes close this one.

Does someone care to explain the difference between this and <C-r>?

C-r will search what you type afterwards, (e.g., "C-r ssh" will then show me the previous command that starts with "ssh") whereas this tip seems to be searching to match what you've already typed. Pretty cool, IMO.

I am using oh-my-zsh and this is already configured for me.

I've been using oh-my-zsh for around a year now and I never realized this.

When I paste this and then source the file, I get:

-bash: \e[A:: command not found -bash: \e[B:: command not found

edit: thanks for the responses below. Learn something new every day :)

It goes in ~/.inputrc, the configuration file for GNU readline. Those aren't shell commands.

.inputrc. And don't source it, just start a new bash.

inputrc is not to be sourced; it's read by readline when the shell starts. You need to restart your shell.

CTRL-x-r will reload it as well, without having to restart your shell

If you really want to put them in .bashrc instead of .inputrc, you can source readline directives with the 'bind' command, like so:

    bind '"\e[A": history-search-backward'   # up-arrow
    bind '"\e[B": history-search-forward'    # down-arrow

Anybody know how to get this to work with Ctrl-p and ctrl-n? I try not to touch the arrow keys because they are too small on my new keyboard :(

Useful. Please share if you have such gems.

Or just use vi mode 'set -o vi' in sh, 'set editing-mode vi' in ~/.inputrc for bash and other readline clients.

This would not allow me to

1) start typing a command;

2) look at or copy+paste stuff from the previous commands

I usually do my history search through history|grep.

Good old doskey

I wouldn't be surprised to learn I'm the only living person still using doskey. Yes, in April 2013, and on W7. I autoload a file with macro definitions in cmd prompt window, and this works great for programs I want at my fingertips but don't want to clutter PATH with (or desktop with shortcuts, or create a .bat/.cmd etc; I do realise there are other ways to do that, but my solution works good enough since DOS times). The unfortunate thing with doskey aliases is you cannot use them as pipe recipient, they have to start at the line start. On the other hand, they can accept parameters.

% exec env zsh

This tip is default behavior in tcsh.

And fish.

And even our old Windows shell...

After cutting my teeth on FreeBSD (with tcsh) the absence/non-defaultedness of this feature always irked me.

I thought that the command history search was default behavior for BASH? (I use zsh)

Or I could use Ctrl-r.

I would say the single most useful thing in bash is the ability to type in commands.

agree, there is (obviously) some key combination that does the same thing without changing the usual behaviour of up/down which I prefer a lot

To be fair, if your prompt is empty, up/down behavior doesn't change.

I guess I re-run commands fairly often, so for me the history-search-backward thing is very useful; when I just need to use the default up/down binding, I just hit Ctrl-E Ctrl-U (go to end of line, erase everything to beginning of line) and I'm good to go.

> To be fair, if your prompt is empty, up/down behavior doesn't change.

Ok, but what if your prompt is empty and you press up once? Will it still continue to maintain the old behaviour?

Yes.

To expand on this, it searches based on what's on the left side of your cursor. So if your cursor is all the way on the left side, it acts as it did before. You lose nothing!

This is indeed very nice.

Unfortunately history-search-(forward|backward) is not mapped to a combination in most common configurations.

I prefer to map it to esc-p which coincides with the setting in tcsh. Like that:

"\M-p": history-search-backward

So you basically type something and then press esc p to have the shell complete it.

I was going to say the ability to Pipe commands but I guess that'd be a bit useless without the ability to type them.

Reply on news.ycombinator.com