Hacker News Reader

Comment by cjauvin | Hacker News Reader

Even though of course nothing beats the power of a set of standardized and ubiquitous tools that you can always count on whenever you find yourself on a new machine, I must admit that I'm favorably inclined toward this trend (or what I perceive as one) of rewriting classic CLI tools with sometimes radical performance improvement promises, but most importantly (for me at least) better UIs: `grep` has always been acceptable for me, but I prefer `ag` these days, and I recently discovered `fd`, which seems like it will replace `find`, as I could never remember its weird syntax, even for the most basic cases.

View on HN

Try rg (ripgrep) over ag:

https://github.com/BurntSushi/ripgrep

ripgrep is incredible

Never managed to switch to ripgrep over ag. Performance difference is not that much of a dealbreaker in practice, and rg is not totally backwards-compatible to ag. I don't exactly remember, what was the problem. I guess it has this annoying practice to ignore everything that's in .gitignore, while for ag I could set up an additional .agignore per-project.

Rg similarly supports it's own ignore file, and optionally disabling the default of respecting .gitignore.

For ripgrep, if you want to use PCRE2 regexes (assuming that's what you meant by backwards compatible), then the -P flag will do that.

Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore.

See the guide for more details: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a...

> Performance difference is not that much of a dealbreaker in practice

This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file:

    $ time rg '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
    3003
    
    real    1.608
    user    1.105
    sys     0.496
    maxmem  9473 MB
    faults  0
    
    $ time ag '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
    ERR: Skipping OpenSubtitles2016.raw.en: pcre_exec() can't handle files larger than 2147483647 bytes.
    
    real    0.016
    user    0.004
    sys     0.011
    maxmem  6 MB
    faults  0

Thank you for your answer.

What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files.

If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve.

One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping.

So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day.

BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).

You want --no-ignore-vcs. There are several other --no-ignore-* flags. ripgrep is a lot more flexible than ag there. And its implementation of the gitignore matching rules should work a lot better than ag. (Search ag's issue tracker to see just how many bugs there are. I'd be surprised if you weren't hitting any of them!)

As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work.

You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.

> --no-ignore-vcs

Oh, thanks, I've missed it somehow.

> rg foo | less -F

Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn }

It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does.

EDIT: actually found a bug report that seems to describe exactly the same problem. https://github.com/BurntSushi/ripgrep/issues/513

Interesting. I can't reproduce that problem. Piping to less works perfectly fine on both my Linux and macOS machines.

If you're willing, I'd definitely appreciate it if you could update #513 with your environment. In particular, by answering these questions: https://github.com/BurntSushi/ripgrep/issues/513#issuecommen...

If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great.

Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.

Is there a way to get the results inline? e.g. first filename, then line number then the matched string? I don't want the filename, newline, matches syntax.

You might want to use the --vimgrep option:

Show results with every match on its own line, including line numbers and column numbers. With this option, a line with more than one match will be printed more than once.

Thanks, this works, I didn't see that option. --no-heading is also useful. (other comment)

My sibling comment suggested --vimgrep, but that will print each match on its own line. For example:

    $ echo foofoo | rg foo --vimgrep
    <stdin>:1:1:foofoo
    <stdin>:1:4:foofoo

What you want is probably `--no-heading`. You can add it to a config file or an alias if you always want it enabled: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#c...

that's it, thank you!

Thank you. Find is so complex and looking at fd is a revelation here.

Now I feel like all my hard earned cli-fu is a lie.

Care to share a link? Using fd as a utility name for an os that already uses the word fd as file descriptor is maddeningly hard to find

I'm assuming it's this - https://github.com/sharkdp/fd

It's the first result for a maddeningly simple search of "cli fd".

I remember posting a bug to WSL about supposedly incorrect behavior with find, only to realize that I had incorrect syntax. I forgot to include the escape on the terminating semicolon (also, conveniently never shown the man pages), or maybe it was the enclosing parens around an or for name pattern matching.

Anyways, Id been away from *nix for a few years at the time, so I was a bit rusty and soon felt like a moron and closed the bug once I realized my error.

> Find is so complex and looking at fd is a revelation here.

> Now I feel like all my hard earned cli-fu is a lie.

fd is also a lie. If you want a decent interface to find just use "find | grep ..."

Shameless plug - I also wrote https://github.com/chmln/sd to replace my use of sed.

> While sed does a whole lot more, sd focuses on doing just one thing and doing it well.

I'm sure you already know this, but sed is meant to provide the features that ed does, so sd is quite a bit more limited. Still, I can see how this can be useful, as most invocations of sed end up only really using a handful of its features.

Yeah, there's no hiding that, the fact that it's a find & replace CLI is declared upfront.

Although personally, I've only really used sed for find & replace so sd solved my gripes with sed perfectly.

alias sd="perl -pe"

Have you tried RipGrep (rg)? I think it's the fastest one these days.

fd is excellent for casual interactive use, i've had a great UX with it and performance feels on par with GNU find.

I don't know if it's because I'm so used to find, but I had a bad UX with fd. This is my thought process each time I use it: "Why separating the extension from the filename? Is this a default? Can we change it? What is the option, -e? Or -e is the option to 'exec'? What's the flag to include hidden files? Or are they already included? Does the pattern comes before the directory? Does it need a directory or does it search it the current one? man fd, man fd, man fd."

I guess my brain just refuses to remember these for some reason.

Right, there's always going to be some additional cognitive overhead with every new tool you adopt. The question is how much you value its benefits for your use case, and is it worth those costs. If you're so used to find, and it works great for you, then keep using it! It's indeed more powerful and standard than fd, which I think is well-suited for vast majority of simple use cases (casual use).

However, here's my reasoning for the first of your questions, as a happy fd user who switched from GNU find. Maybe it can give you insight into why the UX is designed as it is.

The extension is distinguished from the filename because it's a common use case (-e jpg less keystrokes than -iname '*.jpg'), it's semantically separate from the filename (IMO) and therefore should not be treated as part of it. And unlike filenames, extensions most often do not need to be pattern matched. If you want to match multiple extensions, just repeat the -e flag e.g. -e jpg -e png.

Author of fd here. Thank you for your feedback.

If you feel that there is anything we could to to improve fd's UX, it would be great if you could share it on GitHub: https://github.com/sharkdp/fd

> Why separating the extension from the filename? Is this a default

It does not need to be! You can just also just use "fd README.md". Admittedly, the "." will actually search for any character (regex), so if you want to be precise, you need to escape it.

Reply on news.ycombinator.com