Ripgrep is faster than grep, ag, Git grep, ucg, pt, sift (2016)

    alias pf="fzf --preview='less {}' --bind shift-up:preview-page-up,shift-down:preview-page-down"

I've made a video and blog post about it here: https://nickjanetakis.com/blog/customize-fzf-ctrl-t-binding-...

I also made https://nickjanetakis.com/blog/using-fzf-to-preview-text-fil... which covers how to modify fzf's built in CTRL+t shortcut to allow for previews too. CTRL+t is a hotkey driven way to fuzzy match a list of files.

  function frg {
      result=$(rg --ignore-case --color=always --line-number --no-heading "$@" |
        fzf --ansi \
            --color 'hl:-1:underline,hl+:-1:underline:reverse' \
            --delimiter ':' \
            --preview "bat --color=always {1} --theme='Solarized (light)' --highlight-line {2}" \
            --preview-window 'up,60%,border-bottom,+{2}+3/3,~3')
      file=${result%%:*}
      linenumber=$(echo "${result}" | cut -d: -f2)
      if [[ -n "$file" ]]; then
              $EDITOR +"${linenumber}" "$file"
      fi
    }

2. You nerd-sniped me into getting rid of the unnecessary `cut` process :)

  file=${result%%:*}
  line=${result#*:}
  line=${line%%:*}

   git ls-files -m -o --exclude-standard | fzf -m --print0 --preview "git diff {1}" | ....

  GIT_WINDOWS="/mnt/c/Program Files/Git/bin/git.exe"
  GIT_LINUX="/usr/bin/git"
  
  case "$(pwd -P)" in
  /mnt/?/*)
    case "$@" in
    # Needed to fix prompt, but it breaks things like paging, colours, etc
    rev-parse*)
      # running linux git for rev-parse seems faster, even without translating paths
      exec "$GIT_LINUX" "$@"
      ;;
    *)
      exec "$GIT_WINDOWS" -c color.ui=always "$@"
      ;;
    esac
    ;;
  *)
    exec "$GIT_LINUX" "$@"
    ;;
  esac

Yeah, I have a bit of a love-hate relationship with it. But I actually have that with all shells out there. I don't know if it's just me or the shells, or (the most likely I think): a bit of both. But PS is available out of the box and using objects vs plain text is a major win in my book, and even though I still don't know half of the syntax by heart it feels less of an endless fight than other shells. And since I use the shell itself for rather basic things and for the rest only for tools (like shown here), we get along just fine.

This flag is especially convenient if you want to search e.g. .yml and .yaml in one go, or .c and .h in one go, etc.

This suggests your corpora are small. If you have small corpora, then it should be absolutely no surprise that one tool taking 40ms and another taking 20ms will matter for standard interactive usage.

``` ./build.c ```

... and then magic happened.

There are feuds about open source tools all the time. Text editors, Linux distros, shells, programming languages, desktop environments, etc... And ugrep vs ripgrep may be a poster child for C++ vs Rust.

It is not all bad, it drives progress, and it usually stays at a technical level, I've yet to see people killing each others for their choice of command line search tool.

‡ The main commercial product of the ugrep author's company at the time was the gSOAP code generator (it may still be), and that it not only works but makes a reasonably good C and C++ API from WSDL is proof that it is the product of a genius madman. It also allowed you to create both the API and WSDL from a C++-ish header, and both .NET and Java WSDL tools worked perfectly with it. We needed it to work and work it did.

At the time, the generated API was just difficult enough to use that I generated another ~1k lines of code for that project. IIRC, the generated API is sort of handle-based, which requires a slightly different approach than the strict RAII approach we were using. Generating that code was a minor adventure (generating the gSOAP code from the header-ish file, generating doxygen XML from the generated gSOAP code, then generating the wrapper C++ from the doxygen XML).

Like automatic encoding detection and transparently searching UTF-16?

Or simple ways for composing character classes, e.g., `[\pL&&\p{Greek}]` for all codepoints in the Greek script that are letters. Another favorite of mine is `\P{ascii}`, which will search for any codepoint that isn't in the ASCII subset.

Or more sophisticated filtering features that let you automatically respect things like gitignore rules.

Those are all things that ripgrep does that grep does not. So I do not favor this explanation personally.

ripgrep has just about all of the functionality that GNU grep does. I would say the two biggest missing pieces at this point are:

* POSIX locale support. (But this might be a feature[1].)

* Support for "basic" regexes or some equivalent that flips the escaping rules around. i.e., You need to write `\+` to match 1 or more things, where as `+` will just match `+ literally.

Otherwise, ripgrep has unfortunately grown just about as many flags as GNU grep.

[1]: https://github.com/mpv-player/mpv/commit/1e70e82baa9193f6f02...

I was under the impression that grep removed mmap() support because it was slower than normal file i/o

Oh I beg to differ! The blog post goes into this. Here's a simple demonstration using ripgrep 14:

    $ ls -l full.txt
    -rw-rw-r-- 1 andrew users 13113340782 Sep 29 12:30 full.txt

    $ time rg -c --no-mmap 'Clipton' full.txt
    294

    real    1.419
    user    0.539
    sys     0.879
    maxmem  15 MB
    faults  0

    $ time LC_ALL=C grep -c 'Clipton' full.txt
    294

    real    6.911
    user    6.078
    sys     0.829
    maxmem  15 MB
    faults  0

    $ time rg -c --no-mmap 'DMZ|Clipton' full.txt
    1070

    real    1.643
    user    0.747
    sys     0.894
    maxmem  15 MB
    faults  0

    $ time LC_ALL=C grep -E -c 'DMZ|Clipton' full.txt
    1070

    real    8.317
    user    7.384
    sys     0.930
    maxmem  15 MB
    faults  0

Also, you can disable ripgrep's opinions with `-uuu`. It's not designed to just be for code searching. You can use it for normal grepping too. It will even automatically revert to the standard grep line format in shell pipelines.

> so I go back to old fashioned grep

If you do `rg -uuu` then it should search the same stuff grep will. Not sure if that's what you meant though.

The standard way to run "AND" queries is through shell pipelines. That is, `rg foo | rg bar` will only print lines containing both. But composition usually comes with costs. The output reverts to the standard grep format and it doesn't interact nicely with contextual options like -C/--context.

See: https://github.com/BurntSushi/ripgrep/issues/875

https://github.com/BurntSushi/aho-corasick/blob/f227162f7c56...

But no `_mm256_sad_epu8`. What an oddly specific question..?

I imagine a lot of devs have grep preinstalled. In fact, where is grep not installed, now that WSL exists?

Another possibility is that your grep search was hogging up your system's memory. That might make it swap. On my systems which do not have swap enabled but do have overcommit enabled, I experience out-of-memory conditions as my system essentially freezing for some period of time until Linux's OOM-killer kicks in and kills the offending process.

I would say the first is more likely than the second. In order for grep to hog up memory, you need to be searching some pretty specific kinds of files. A simple log file probably won't do it. But... a big binary file? Sure:

    grep -a burntsushi /proc/self/pagemap

(There are other reasons for a system to lock up, but the above two are the ones that are pretty common for me. Well, in the past anyway. Now my machines have oodles of RAM and lots of I/O bandwidth.)

Source? Ripgrep's benchmarks show it significantly faster.

One practical result of this is that it will mean `ack` will be quite slow when searching typical checkouts of Node.js or Rust projects, because it won't automatically ignore the `node_modules` or `target` directories. In both cases, those directories can become enormous.

`ack` will ignore things like `.git` by default though.

I believe `ag` was the first widely used grep-like tool that attempted to respect your .gitignore files automatically. (Besides, of course, `git grep`. But `git grep` behaves a little differently. It only searches what is tracked in the repo, and that may or may not be in sync with the rules in your gitignores.)