FZF and RipGrep – Navigate with bash faster than ever before | Hacker News Reader

FZF and RipGrep – Navigate with bash faster than ever before

owen.cymru

171 pointsMidasx53 comments

For dev work, I still prefer cscope for searching for things, at least in all supported languages. cscope is _fantastic_. Combined with a $CSCOPE_EDITOR that does the right thing (for me: open $EDITOR in the background in a new tmux window titled after the file being edited, returning control to cscope for additional searches / opens).

What's really good about cscope is this:

- multiple search types such as: where is this symbol assigned to, what functions call this function, where is this defined, where is this referenced, and so on

- curses and CLI interface

- the curses interface is very nice

- the CLI interface is very useful for integration into your $EDITOR, whatever it might be

The only downside is that cscope only supports C-like languages and a few others like yacc. So this works for C, C++, and Java (to a lesser extent), but does not work at all for languages like, say, Python.

These look awesome! If you're willing to put in that much effort to learn new tools though, could also try zsh or fish. The built-in autocomplete of both over bash is on another level.

There are good reasons to use Zsh beyond its autocomplete system. Zsh parameter expansions are a beautiful thing. When I write CLI scripts, I don't write Bash scripts, I write Zsh scripts.

Documentation, however, is lacking. It's not quite as bad as Vim, but there are massive swaths of the program, entire subsystems, that are more or less completely undocumented. It's very user unfriendly.

Also, the ad-hoc "plugin" ecosystem is in a strange and fragmented state. It's a shame that more people don't write actual modules in C (yes, Zsh has a native module system), but then again, is the API even documented?

The thing about Fish vs Zsh is that Zsh, with all those pre-command hooks running in Zsh scripts to replicate out-of-the-box Fish functionality, it's slow. I haven't tried compiling my plugin files (yes, Zsh also has a byte compiler), so maybe that would help. But I've occasionally found myself poring over logs from zsh -x to see diagose my 200ms delay.

One more thing about Zsh: it has emacs syndrome. Do your really need an FTP client and calendar built into your shell?

I've been using zsh for a long time and I didn't know it had native plugins, a bytecode compiler, an FTP client and a calendar.

To be frank, that's kind of scary, I'll probably have to take another look at fish :)

Zsh was undocumented in Ubuntu 14 because a bug in the package prevented the manuals from being installed and due to the way Canonical handles packages it was never fixed for the entirety of Ubuntu 14.

I am one of those few remaining Zsh users that doesn't have an 200MB baggage train on my shell though. I have a modest (2kb) .zshrc/.zshenv and for the most part just use the builtins.

> There are good reasons to use Zsh beyond its autocomplete system...

zsh also performs much better in practice than bash. Not only does it run the same code faster, it subsumes a great deal of functionality that normally requires external utility calls (which are very slow) into the shell itself.

For example:

* zsh has native floating-point arithmetic, so no need for `awk` or whatever. It can even format (digit-group) the numbers

* Regular-expression matching with =~ can be replaced by extended globs (which offer similar functionality but are significantly faster in most cases)

* `basename`, `dirname`, `readlink -m`, and `readlink -f` can be replaced by parameter-expansion modifiers

* `sort` can be replaced by parameter-expansion flags

* `grep` can be replaced by parameter-expansion flags and extended globs

* `find` (including its `-type` and `-exec` features) can be replaced by globs

* `date` can be replaced by prompt expansion or the `zsh/datetime` module

* `cp`, `mkdir`, `rm`, and so on can be replaced by the `zsh/files` module

* `stat` can be replaced by the `zsh/stat` module

* `column` can be replaced by `print -c` or `print -C`

> there are massive swaths of the program, entire subsystems, that are more or less completely undocumented

Like what? I'm not sure i'd noticed that myself. Sometimes the documentation is vague, maybe hard to find (e.g., the completion system's documentation is a bit overwhelming), but it's always been there when i went looking for it.

> Also, the ad-hoc "plugin" ecosystem is in a strange and fragmented state.

The plug-in ecosystem (i assume you mean OMZ, zplug, and stuff like that) is entirely unofficial and most of the zsh developers don't seem to care for any of it because it tends to be slow, error-prone, and often just unnecessary. It's also a major support burden for the people on IRC and in the mailing list, which doesn't endear them to it. Might be cool to have something official though.

> It's a shame that more people don't write actual modules in C (yes, Zsh has a native module system), but then again, is the API even documented?

I don't think it is, there's just an example module you can build. bash also supports loadable modules and it's the same way AFAIK.

> One more thing about Zsh: it has emacs syndrome. Do your really need an FTP client and calendar built into your shell?

Most of the weird stuff like the FTP client and calendar and Tetris game are optional modules or even just regular shell scripts. None of them are enabled by default and packagers can omit them (and they often do, especially with static builds).

> zsh also performs much better in practice than bash. Not only does it run the same code faster, it subsumes a great deal of functionality that normally requires external utility calls (which are very slow) into the shell itself.

For the uninitiated: Zsh parameters expansions and globbing (called "filename generation" in the docs) are beautiful. As I said above, I don't write Bash scripts, I write Zsh scripts, and this kind of thing is why:

    % cd /tmp
    % mkdir example
    % cd example
    % touch foo
    % touch bar
    % mkdir "my documents"
    % touch "my documents/baz"

    % tree -N
    .
    ├── bar
    ├── foo
    └── my documents
        └── baz

    1 directory, 3 files

    % for f in **/*(.);
    >   do echo File ${f:t} lives in ${f:A:h}
    > done
    File bar lives in /tmp/example
    File foo lives in /tmp/example
    File baz lives in /tmp/example/my documents

> Like what? I'm not sure i'd noticed that myself. Sometimes the documentation is vague, maybe hard to find (e.g., the completion system's documentation is a bit overwhelming), but it's always been there when i went looking for it.

Fair enough. Although IMO "vague and disorganized" is just as good as "nonexistent". Just try reading the docs for `autoload`/`typeset`/`local`, or `zstyle` [0], or `zmodload`. The whole thing badly needs to be tagged and reverse-indexed by functionality ("how do I accomplish X"), not by flag ("what does foo -q mean?"). I wrote a long comment about this kind of documentation recently [1].

> The plug-in ecosystem (i assume you mean OMZ, zplug, and stuff like that) is entirely unofficial and most of the zsh developers don't seem to care for any of it because it tends to be slow, error-prone, and often just unnecessary. It's also a major support burden for the people on IRC and in the mailing list, which doesn't endear them to it. Might be cool to have something official though.

Yes, this. It seems like everyone and their mother at one point decided to try to write a "package manager" for Zsh.

But I think the problem with these unofficial plugins is mainly that sourcing thousands of lines of shell script on every terminal load, and running potentially dozens of functions before every command, is just plain slow. Zsh might be faster than Bash, but it's still really really slow. Just looping over 1..1000000 takes well over a second in Zsh, but less than 1/2 second in Python (which is already considered a slow language).

What I haven't tried is zcompile-ing all my startup scripts. Would that help?

And of course, actually documenting the C API and build process would be a good first step for the maintainers and devs. The example module is easy enough to follow, but actually writing a module is another matter entirely. This is especially frustrating because AFAICT Zsh builtins are themselves implemented as a statically-linked module called "zsh/main", so it must be a powerful system.

[0]: https://unix.stackexchange.com/q/214657

[1]: https://news.ycombinator.com/item?id=15479273

I added ctrl-P bind to open with vim. But if I realize I didn't want to do that and want to cancel then hitting ESC or ctrl-c just drops me into vim with no file. Any ideas on how to change that bind so that I can cancel back to the command line?

Edit: actually it's even worse, it seems like rg keeps running in the background even after I ctrl-c, which if I'm starting in some directory at the top of my directory tree means it runs for a long time.

Edit 2: Looks like the problem in my edit is a known issue with ripgrep: https://github.com/BurntSushi/ripgrep/issues/200

Sorry, I should update the article with the better solution, it's just a few more lines:

https://github.com/bag-man/dotfiles/blob/master/bashrc#L116-...

Yeah, that worked, thank you!

It is also cool to add preview

https://gist.github.com/chew-z/db310b0e3184ae335e24d2ec7087a...

I am using ag. Is it worth changing to RipGrep ?

The other unix command line tool that I love is autojump.

I am thinking on trying fish and probably fzf.

RipGrep doesn't have some very useful PCRE features.

https://github.com/BurntSushi/ripgrep#why-shouldnt-i-use-rip...

If you use lookarounds (or often use grep -P), RipGrep may not cover all of your needed use cases.

That said, it's very fast for general use.

You might find this discussion between the creators of The Silver Searcher and ripgrep useful:

https://news.ycombinator.com/item?id=12567328

FZF and fish don't play well if I remember correctly (I hope I'm mistaken though.)

I was just looking at fish.

It has a plugin system called fisherman. With plugins fzf, and z - An autojump clone.

https://github.com/fisherman/fzf

https://github.com/fisherman/z

Fish user here! FZF works beautifully with fish, no plugin manager needed. `brew install fzf` gives you the option to install scripts for fish if it's your shell (IIRC).

I wanted to try fish too, but too many plugins or libs has a warning on github "does not work with fish" so I just saved myself some headaches and decided to resume with using already great zsh.

I use both ag and rg with fish. For me, both are equally fast (because my repositories are small enough that it doesn't matter), but ag has better UI (I remember the --python option, but have no idea how to filter with rg). Those are about the only differences that matter to me, so I use whatever is muscle memory. Usually rg, unless I want to search for specific filetypes.

If you're wondering whether to switch, don't bother unless you find ag slow.

Fish, on the other hand, I can't live without. I recommend fisherman and z, and a few other plugins I forget now (I use one that shows a notification if your long running command completes while the terminal is in the background, I love it).

> don't bother unless you find ag slow

Or if you want more correct gitignore matching.

> (I remember the --python option, but have no idea how to filter with rg)

    rg -tpy 'def __init__'

or, to exclude Python files

    rg -Tpy 'def __init__'

You can see the list of types available with `rg --type-list`.

Ah, thanks for that, I didn't even know rg had types (I thought -t accepted the actual extensions). I'm not saying your tool is worse than ag, just that it doesn't make sense to switch if you're not having problems with ag.

I love the combination of these! Really opens up a world of possibilities. The one command I'm still trying to replace is `cd` itself. For instance, can anyone suggest a way to add bookmarks, so that I can move around faster? I found a few, but was never happy with them.

I’ve been enjoying autojump [1], it automagically creates bookmarks and prioritises them based on frequency of visiting said folders.

[1] https://github.com/wting/autojump

Autojump is the answer here. Totally a gamechanger. If you like its semantics enough, check out fasd [1] , which is a superset of autojump that applies its cache in many other places. Absolutely awesome.

[1] https://github.com/clvv/fasd

Scratched that itch of mine for bash/zsh in 30 lines[0]. kd stands for quicKDir or worKDir depending on the moon.

    kd foobar ~/Workspace/foobar  # define bookmark
    cd ~/some/where
    kd foobar                     # goes to bookmark
    kd foo                        # matches on string prefix, last entry wins
    
    cd ~/Workspace/quuuuuux
    kd qux "$PWD"                 # quick bookmark current dir
    
    touch Gemfile                 # define project root
    cd app/controllers/name/space/whatevs
    kd                            # goes to project root

Turns out autojump doesn't work for me, too many projects in parallel. Since this one is manual, it can be very semantic. Those 30 lines saved me hours on end of tab mashing, especially in Go workspaces.

[0]: https://github.com/lloeki/dotfiles/blob/master/shell/kd

http://cryptonector.com/2007/04/c-shell-pushdpopd-on-steroid...

I have been doing something similar for many years. Someone ported this to Bash... I'll try to get that port up on github.

EDIT: https://raw.githubusercontent.com/jakobi/script-archive/mast...

I recommend https://github.com/clvv/fasd It basically makes bookmarks automatically as you navigate your filesystem anyway.

https://github.com/rupa/z could also be interesting for you.

Also interesting is fzf-marks:

https://github.com/urbainvaes/fzf-marks.git

It isn't super clear in the article, but I now use bfs & fzf for replacing cd.

https://github.com/bag-man/dotfiles/blob/master/bashrc#L73

Lets you change directory with alt+c, although I have tweaked it so you always search from your root folder. Which isn't for everyone. Otherwise fzf alt+c doesn't let you go back up directories which gets frustrating.

I've just tried this and in my case it leads to "cd \~[...]" which obviously does not work.

That's strange. Maybe just change it to:

    export FZF_ALT_C_COMMAND="bfs -type d -nohidden"

(and install bfs of course)

It won't let you go back up the tree, but if you are clever you can use `cd -` to jump back to your previous location.

Hey, bfs author here, glad you've found it useful! You can do that whole thing in one command with

    bfs ~ -nohidden -type d -printf '~/%P\n'

I use a combination of https://github.com/clvv/fasd and https://github.com/ranger/ranger.

When tasks like navigating into a project, starting tests, local server and neovim become repetitive I usually end up making a tmux script which does everything for me.

Also, aliasing ‘..’ to ‘cd ..’ and such.

> The one command I'm still trying to replace is `cd` itself.

zsh will implicitly cd if you just mention a directory e.g. `$ ../foo/bar [RET]` will cd there without mention. These days I only use `cd -`.

IIRC that isn't the default. You need to enable the option for that to work.

zsh has about a thousand options you can configure.

> IIRC that isn't the default. You need to enable the option for that to work.

You're right, it's been so long I completely forgot.

It's auto_cd, and auto_pushd will also automatically push new directories to the dstack.

> zsh has about a thousand options you can configure.

I should probably go through the options list in its entirety one day, but technically according to http://zsh.sourceforge.net/Doc/Release/Options.html it's closer to a quarter of that ;)

I think it is enabled by default. I never explicitly enabled it for my zsh configuration and it behaves that way.

> These days I only use `cd -`.

There's also `cd foo bar` in zsh, which substitutes foo with bar in the name of $PWD. I rarely used it, and when I do, it's usually for a well known pair of lengthy named dirs, yet it also shaves off a couple of keystrokes pretty much daily for me.

The $CDPATH variable (built into `cd`)[0] plus the `bash_completion`[1] script to enable tab-completion of CDPATH dirs - works for my use case (FWIW).

[0] http://linuxcommand.org/lc3_man_pages/cdh.html [1] http://www.caliban.org/bash/index.shtml#completion

Probably not exactly what you want, but the pushd/popd/dirs commands have existed in all major shells forever.

You can run dirs -v to get the list, and do cd +(num) to go there.

So:

  FreeBSD <jubei/pts/2> (219 /var): dirs -v
  0	/var
  1	/usr/local
  2	~
  FreeBSD <jubei/pts/2> (220 /var): cd +1
  /usr/local
  FreeBSD <jubei/pts/2> (221 /usr/local): _

I have a little script that combines z and fzf.

https://gist.github.com/chew-z/44f3bcdc08eaecdf306868f9a3d0e...

If you like fzf you might be interested in fzy:

https://github.com/jhawthorn/fzy

The second post i read today about fzf. And also again i see ripgrep. I think it is time to make these tools a default in linux distros. Maybe a small you could use ripgrep instead of grep, just before the first use of these tools would be interesting.

Hope you don't mind some brief clarifications.

Work on adding ripgrep to Linux distros is being tracked here[1]. We've made good progress so far, but AFAIK it's still missing in Ubuntu and Debian. I'm not caught up with what's required to get ripgrep into those repos.

With respect to making it a literal default---as in replacing GNU grep---I don't really expect that to ever happen. There is a ton of intersection between the tools, right down to the names and functionality of flags, but there's also many subtle details in the differences. For example, normal grep invocations will use BREs by default, which have different escaping rules than EREs, where EREs are closer to what ripgrep uses. There's also the difference where ripgrep respects things like .gitignore and ignores hidden files by default, which means there's likely a non-zero number of shell scripts out there where swapping grep for ripgrep will break. Folks won't (and shouldn't) take too kindly to that. :-)

To a first approximation, ripgrep is optimized for end user experience in a terminal. This leads to different design decisions. Offering a compatibility mode with grep has been suggested, but is significant work.

[1] - https://github.com/BurntSushi/ripgrep/issues/10

Have you set up a Hackernews/Reddit crawler that pings you each time someone mentions "ripgrep"? Seriously, I'd be interested in knowing what solution you use.

I just saw it on the front page, which I check embarrassingly frequently. :-) (But I do have a Google Alert setup for it, sure.)

HN Search, at the bottom of the page, is very useful for keeping tabs on discussions of topics you care about.

> Work on adding ripgrep to Linux distros is being tracked here[1]

There are multiple interpretations of the GP's comment, each one being a prerequisite of the next:

- ripgrep is available in the distro repos - ripgrep is included on the main install medium (first Debian CD, Arch net iso even?) - ripgrep is selected for inclusion on a default install (but grep is still there) - ripgrep replaces grep altogether

On first read I though we were talking about the third one.

> where swapping grep for ripgrep will break

> Offering a compatibility mode with grep has been suggested, but is significant work.

Do you mean by that, that 'rg' could change its defaults and behave like 'grep' when being invoked as 'grep' (through symlink or hardlink), similarly to bash/sh or vi/vim?

I agree with the multiple possible interpretations. I conservatively chose the most controversial one and responded to that, which should cover all of them. :-)

RE compatibility: yes, something like that. But "change its defaults" is the easy part. I said more here: https://news.ycombinator.com/item?id=15430770

The most effective way into “official” Ubuntu is probably via Debian, although there's nothing stopping you from providing your own Ubuntu PPA.

If you want to get a ripgrep package into Debian then you’ll need to do the work to turn the sources into something that Debian can build automatically & find yourself a Debian developer who’ll act as a sponsor for your package and upload it to the Debian archive.

The Debian Mentors mailing list is probably a good place to start, although you could make a sponsor request directly via the bug tracking system if you wanted to:

https://wiki.debian.org/DebianMentorsFaq#How_do_I_get_a_spon...

How does fzf actually work? What string matching techniques are behind it?

their docs explain a lot on how it works, all the options and etc https://github.com/junegunn/fzf

Reply on news.ycombinator.com