Ripgrep 14 Released | Hacker News Reader

Ripgrep 14 Released

github.com

297 pointstimf71 comments

Andrew sets an extremely high bar for open-source engineering. As a loose member of the broader Rust community, I've benefited enormously not only from his code but also from his discussion comments and his blog posts (especially [1] and [2]).

Congratulations on the release, Andrew! I also notice that you've joined Astral [3], which as a fan of Rust/Ruff/rg I'm thrilled about.

[1]: https://blog.burntsushi.net/transducers/

[2]: https://blog.burntsushi.net/foss/

[3]: https://github.com/astral-sh

There is a distinct joy in showing someone rg who has never seen it before and then seeing them immediately adopt it as a daily tool.

Recently a colleague had a bug. He told me it was related to the "X" that was weirdly behaving like a "Y".

I fd'd "X" and then rg'd the resulting files for "Y" and found a place where some copy/pasted code was treating "X" as a "Y". Big monorepo codebase, absolutely just tore through it and solved the entire bug.

This happened to me. The codebase I work with is hundreds of gigabytes and I was using find --exec grep, and my coworker showed me rg and I couldn't believe what I was seeing. It's a beautiful and amazing tool.

Since Visual Studio Code is using ripgrep for search one can also open the repo in the editor and use find in files to search for such things similar performance. It’s one of my favorite tools for searching specific expressions in codebases.

That's why I initially jumped on the VSCode bandwagon. I'm slightly miffed that nowadays the search is much slower than ripgrep. I see myself using ripgrep from the VSCode console.

I don't think it's ripgrep itself, it's something with the way VSCode manages extensions, and doesn't give you enough information to debug which extension is blocking it.

I always just find it faster to put a command argument together tbh

If you are an Emacs user like me, you must try out the consult-ripgrep command from the peerless Consult [1] package by Daniel Mendler: search your whole project with ripgrep and get a live preview of every matching candidate all inside of Emacs!

[1]: https://github.com/minad/consult

am a big fan of the whole consult/vertico/embark stack of emacs plugins. They're fantastic.

En garde! If you're a Vim user, fzf.vim [1] can do this. :)

[1]: https://github.com/junegunn/fzf.vim

And if you want interactive support, try https://github.com/bombela/fzf.vim.rgfd

(shameless plug)

If you don't need live updating output, you can just set grepprg and get results in the quickfix list with :grep[!] (or the location list with :lgrep[!]).

  if executable('rg')
    let &grepprg = 'rg --vimgrep $*'
  endif

No need for Fzf/Telescope/Denite/DDU or anything else in that case.

See:

https://neovim.io/doc/user/quickfix.html#%3Agrep

https://neovim.io/doc/user/options.html#'grepprg'

used to do that with glimpse when i worked with very large codebases 23 years ago!

big fan of ag, ripgrep and burntsushi's rust work!

And for those who are also on Emacs but on the Swiper/avy/ivy/counsel side of the fence, there's counsel-rg.

I use it since before burntsushi (who's here on HN btw)'s ripgrep was shipped with Debian stable!

> search your whole project with ripgrep and get a live preview of every matching candidate all inside of Emacs

I'm sometimes searching not just my project but my entire user dir or my entire shared drive, from Emacs. A NVMe PCIe 4.0 SSD (I'm using a WD SN850X which someone here recommended to me when I assembled my PC) is that fast and ripgrep too.

> Swiper/avy/ivy/counsel

What are these? They're not editors/IDEs.. or?

Oh, they are Emacs extensions.

Ah, thank you. My brain inserted a "not" in place of "also", so I read:

> And for those who are also on Emacs but on the Swiper/avy/ivy/counsel side of the fence

As "And for those who are not on Emacs but..."

deadgrep is nice too

Yep, the consult suite and rg are two of my favorite things. And I'm not even a SWE.

Ripgrep is probably my favorite command line tool (which replaces older solutions). It’s just so quick to search a folder for a specific line of text in a file.

Same here... the program I use most aside from my editor.

In fact my current editor, VS Code, also does searching powered by rg behind the scenes:

  /Applications/Visual\ Studio\ Code.app/Contents/Resources/app/node_modules.asar.unpacked/vscode-ripgrep/bin/rg

My muscle memory is still dialed in to `ack`, but I recently built an open source tool called `grep-ast` [0] that serves a similar function to ripgrep, ack, etc. The difference is that it shows matching lines in the context of the functions/methods/classes/etc that contain them.

It uses the abstract syntax tree (AST) of the source code to show how the matching lines fit into the code structure. It shows relevant code from every layer of the AST, above and below the matches.

It feels quite useful when you're grepping to understand how functions, classes, variables etc are used within a non-trivial codebase.

Here's a snippet that shows grep-ast searching the django repo. Notice that it finds `ROOT_URLCONF` and then shows you the method and class that contain the matching line, including a helpful part of the docstring. If you ran this in the terminal, it would also colorize the matches.

  django$ gast ROOT_URLCONF

  middleware/locale.py:

  │from django.conf import settings
  │from django.conf.urls.i18n import is_language_prefix_patterns_used
  │from django.http import HttpResponseRedirect
  ⋮...
  │class LocaleMiddleware(MiddlewareMixin):
  │    """
  │    Parse a request and decide what translation object to install in the
  │    current thread context. This allows pages to be dynamically translated to
  │    the language the user desires (if the language is available).
  ⋮...
  │    def process_request(self, request):
  ▶        urlconf = getattr(request, "urlconf", settings.ROOT_URLCONF)

[0] https://github.com/paul-gauthier/grep-ast

have you looked at semgrep? This isn't the same thing but is similarly poking at the whole "ASTs are a thing we want to grep at" problem

Thanks for the pointer. There are definitely a few tools that have explored the idea of searching the AST of code. Semgrep seems to do that, as does a tool called ast-grep [0].

Both of them are sort of doing the opposite of my tool. They are letting you specify your search as a chunk of code/AST.

My tool let's you grep a regex as usual, but shows you the matches in a helpful AST aware way.

[0] https://ast-grep.github.io/

Thanks for making this. Immediately useful.

rg is great, I use it a lot. Recently I have also used [ambr](https://github.com/dalance/amber) which can do both search (ambs) and replace (ambr) recursively in your codebase. The only problem as of yet is that it does not support globbing so I cannot filter on certain filetypes only.

You could use "find" (or even better, "fd") to find specific filetypes, then pass it to amber via xargs or some similar way.

I'll throw in sd as a nice sed/find-and-replace tool. Using fd + xargs + sd is a pretty good workflow if a shell glob isn't good enough to target the files you want. https://github.com/chmln/sd

I’ll also throw in Leah Neukirche ‘s xe as a better alternative to xargs: https://github.com/leahneukirchen/xe

I wanted to like sd but it doesn't support my main use case of recursive search/replace. Imagine if every time you wanted to grep some files you had to build a find -print0 | xargs | rg pipeline... it just takes me out of the flow too much. I'm glad people are posting other options here, I'm looking forward to trying them.

https://github.com/chmln/sd/issues/62

If you haven't discovered recursive path expansion with `**` yet, which is supported by a number of popular shells, including bash, it is about to improve your shell life.

Agreed, doing sd 'search' 'replace' **.py is common in my history. I only mention fd + xargs as a backup for when you need to do something that a simple shell expansion won't cover.

Also, for a "confirm your changes" type workflow, I like git add -p after running the sd command just to review all of the changes.

How does that work?

It's glob and/or bsd_glob from the underlying C stdlib. Another interesting example of things the stdlib glob can do:

  #!/usr/bin/perl
  # perl since it's an easy way to use an arbitrary function from stdlib
  use File::Glob "bsd_glob";
  for (bsd_glob("{Hello,Hi} {world,everyone}...a {demo,example}")) {
    print($_."\n");
  }

https://github.com/facebookincubator/fastmod is also great for the replace usecase.

This use case is a killer feature of emacs. Rg supports wgrep (writable grep). `rg` to match lines, and then in the resulting buffer, `e` to edit the results and save the changes back to the matched files.

See also ast-grep

https://github.com/ast-grep/ast-grep/

> ast-grep is a AST-based tool to search code by pattern code. Think it as your old-friend grep but it matches AST nodes instead of text. You can write patterns as if you are writing ordinary code. It will match all code that has the same syntactical structure. You can use $ sign + upper case letters as wildcard, e.g. $MATCH, to match any single AST node. Think it as REGEX dot ., except it is not textual.

I use Ripgrep daily, and it helps me a lot when navigating codebases, and I even use it to search through my notes. A while ago, I made a script that looks for file paths in the output from Ripgrep in order to turn the paths into clickable links, as I couldn't get it to work properly back then. I'm so happy to hear that «the headlining feature in this release is hyperlink support»! I'm really looking forward to using the new update.

I’m not an rg wizard by any means, but it is one of the few utilities that I end up using daily for one reason or another.

The notes state that the headline feature is "hyperlink support". However, the notes don't seem to really explain what that means. Can someone explain a bit more about what that feature does/is? What's a use case?

It creates links in the output. You click them. It opens the file. That's pretty much it.

Terminals (some terminals) support hyperlinks and it creates clickable links. For example ls -l --hyperlink=auto If you terminal supports it, you can click the names.

See here for more info: https://github.com/Alhadis/OSC8-Adoption/

So, when you search for "foo" in mydir/, rg can find the term foo inside different places in a number of files, and then print the results like:

  mydir/myfile.java
  15: return foo;
  78: System.out.println(foo);
  123: // TODO: change foo to bar

Hyperlink support means that the line numbers (15, 78, 123) are clickable, and will open your favorite editor to that file and that line number.

That's if your terminal supports hyperlinks, or has hyperlink support enabled - most do. Depending on the terminal app it might be control/cmd click, or option-click to open the hyperlink.

Note (this got me while trying): hyperlinks are not emitted when you search inside a specific file directly, e.g. "rg foo myfile.py".

I'm fascinated! I've been living with and authoring CLI tools for about a decade now and I didn't know that there's widespread support for additional attributes on text besides styling (color, italics, underline, etc) in common terminal emulators. What a cool thing!

For the single file case, if you pass --with-filename then it should work.

Sibling comments have explained, but the keyword to search to learn more is "OSC-8 hyperlinks"

Extra life hack for Mac users — you can use mdfind first and then pipe those results into rg with xargs. Even faster. Mdfind has an index already so the results are almost instant even across huge numbers of files. Major qol improvement.

Thanks for the pointer - I was unaware of:

https://ss64.com/osx/mdfind.html

I'm still sad that --sort-files makes ripgrep run in single-core mode. (I know you can't make --sort-files _free_ in multi-core mode, but it would still be faster than single-core)

https://github.com/BurntSushi/ripgrep/issues/152

Seems like the issue pretty much covers this.

The shell function from this comment works pretty well for me: https://github.com/BurntSushi/ripgrep/issues/152#issuecommen...

I'm very thankful for ripgrep, ag and fzf.

I don't seem to get the hyperlinks with either Terminal or iTerm.

P.S. OK found it: it can't show hyperlinks when you explicitly pass a file name to search (for internal implementation reasons, might be fixed later). It works when you search in directories etc.

Hyperlink support? Hell yes! This has been such a frustration of using terminal tools to search in files. I love that the vscode format even linkifies by matched line. Thanks for this update burntsushi!

What does « hyperlink support » mean? The release notes don’t give details, except that it’s supported.

You have one explanation here: https://wezfurlong.org/wezterm/hyperlinks.html#explicit-hype...

Thanks! I didn’t realize there was an escape sequence for hyperlinks

Ripgrep is an essential tool for navigating unfamiliar codebases. I also use it to quickly search many repositories for specific symbols when I research the consumers of libraries and APIs.

What's the improvement this time? Translate DFAs to microcode?

https://blog.burntsushi.net/regex-internals/

I appreciated the Sherlock Holmes joke:

> $ regex-cli find match pikevm --no-table -p '\b\w+\b' -y 'Σέρλοκ Χολμς'

0:0:12:Σέρλοκ

0:13:23:Χολμς

so... its grep to ack to ag to rg, is there something next?

git to tig

ls to exa

Cat to bat

find -> fd

The release notes use the word "headling" twice instead of presumably "headline". I had to look it up in case this was a new usage of this archaic word.

Ah nice catch! I meant to say "headlining." Fixed now.

I hope you used rg to catch the catch :)

#eatyourowndogfood

There was also ack (IIRC) written in Perl, earlier.

I wish Jetbrains would integrate ripgrep into Pycharm as a native feature.

Reply on news.ycombinator.com