There, hyperscan is generally the king, which means hypergrep numbers are likely accurate: https://github.com/p-ranav/hypergrep?tab=readme-ov-file#dire...
Disclaimer: I rarely use any *grep utilities, but often implement string libraries.
Your claim is true to a first approximation. But greps are line oriented, and that means there are optimizations that can be done that are hard to do in a general regex library. You can read more about that here: https://blog.burntsushi.net/ripgrep/#anatomy-of-a-grep (greps are more than simple CLI wrappers around a regex engine).
If you read my commentary in the ripgrep discussion above, you'll note that it isn't just about the benchmarks themselves being accurate, but the model they represent. Nevertheless, I linked the hypergrep benchmarks not because of Hyperscan, but because they were done by someone who isn't the author of either ripgrep or ugrep.
As for regex benchmarks, you'll want to check out rebar: https://github.com/BurntSushi/rebar
You can see my full thoughts around benchmark design and philosophy if you read the rebar documentation. Be warned though, you'll need some time.
There is a fork of ripgrep with Hyperscan support: https://sr.ht/~pierrenn/ripgrep/
Hyperscan also has some preculiarities on how it reports matches. You won't notice it in basic usage, but it will appear when using something like the -o/--only-matching flag. For example, Hyperscan will report matches of a, b and c for the regex \w+, where as a normal grep will just report a match of abc. (And this makes sense given the design and motivation for Hyperscan.) Hypergrep goes to some pain to paper over this, but IIRC the logic is not fully correct. I'm on mobile, otherwise I would link to the reddit thread where I had a convo about this with the hypergrep author.
From some searching I think you might mean this: https://www.reddit.com/r/cpp/comments/143d148/hypergrep_a_ne...
$ echo 'foobar' | hg -o '\w{3}'
1:foobar
$ echo 'foobar' | grep -E -n -o '\w{3}'
1:foo
1:bar
I want to be clear that these are intended semantics as part of Hyperscan. It's not a bug with Hyperscan. But it is something you'll need to figure out how to deal with (whether that's papering over it somehow, although I'm not sure that's possible, or documenting it as a difference) if you're building a grep around Hyperscan.
I also wonder if there's a performance issue when matching a really long line because Hyperscan is not greedy and will ping back to Hypergrep for every sub match. I guessing this is the reason for those shenanigans in the callback [0].
$ python -c 'print("foo" + "bar" * 3000)' | hg -o 'foo.*bar'
It's hard to learn too much from hypergrep. It still has some rough spots:
$ hgrep -o 'foo.*bar' foobarbar.txt
foobarbar.txt
1:[Omitted long line with 1 matches]
$ hgrep -M0 -o 'foo.*bar' foobarbar.txt
Too few arguments
For more information try --help
$ hgrep -M 0 -o 'foo.*bar' foobarbar.txt
foobarbar.txt
1:[Omitted long line with 1 matches]
$ hgrep -M 0 'foo.*bar' foobarbar.txt
foobarbar.txt
1:[Omitted long line with 1 matches]
$ hgrep -M0 'foo.*bar' foobarbar.txt
terminate called after throwing an instance of 'std::invalid_argument'
what(): pattern not found
zsh: IOT instruction (core dumped) hgrep -M0 'foo.*bar' foobarbar.txt
> This flag instructs Hyperscan to treat the pattern as a sequence of UTF-8 characters. The results of scanning invalid UTF-8 sequences with a Hyperscan library that has been compiled with one or more patterns using this flag are undefined.
That's another issue you'll need to grapple with if you use Hyperscan. PCRE2 used to have this issue[4], but they've since defined the semantics of searching invalid UTF-8 with Unicode mode enabled. ripgrep 14 uses that new mode, but I haven't updated that FAQ answer yet.
Hyperscan isn't alone. Many regex engines do not support searching arbitrary byte sequences[5]. And this is why many/most regex engines are awkward to use in a fast grep implementation. Because you really do not want your grep to fall over when it comes across invalid UTF-8. And the overhead of doing UTF-8 checking in the first place (and perhaps let you just skip over lines that contain invalid UTF-8) would make it difficult to be competitive in performance. It also inhibits its usage in OSINT work.
[1]: https://intel.github.io/hyperscan/dev-reference/api_files.ht...
[2]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a...
[3]: https://github.com/p-ranav/hypergrep/blob/ee85b713aa84e0050a...
[4]: https://github.com/BurntSushi/ripgrep/blob/master/FAQ.md#why...
[5]: https://github.com/BurntSushi/rebar/blob/96c6779b7e1cdd850b8...
You could perhaps work around this by only applying it as an optimization when you know the pattern has identical semantics in both regex engines. But you would have to do the work to characterize them.
I would rather just make the regex crate faster. If you look at the rebar benchmarks, it's not that far behind and is sometimes even faster. The case where Hyperscan really destroys everything else is for searches for many patterns.
Hyperscan has other logistical issues. It is a beast to build. And its pattern compilation times can be large (again, see rebar). Hyperscan itself only supports x86-64, so one would probably want to actually use Vectorscan (a fork of Hyperscan that supports additional architectures).