Comment by librasteve | Hacker News Reader

Comment by librasteve | Hacker News Reader

Parent Full thread

@burntsushi - thanks for the benchmarks - certainly rg is amazingly fast! I was wondering if somehow rak and rg could be combined to get the benefit of both worlds (eg. use the strong ffi support in raku and rust to interface them) ... but this would only make any sense if you could apply the rg speed boost to raku regex and thus get the syntax benefits too

(sidebar - Larry took to opportunity with perl6/raku to properly reinvent regexes for unicode - there is quite lot to unpack in raku regexes: graphemes management, unicode props as char classes, named regexes, etc.)

soo - I suggest this proposal to anyone looking for a cool rust/unicode project ... extend ripgrep from PCRE/ECMA262 regex 1.0 to embrace the "21st century" regex syntax of raku

some initial reading is here: - https://dev.to/bbkr/utf-8-glyphs-and-graphemes-331b - https://docs.raku.org/language/regexes - https://github.com/edumentab/p6-ecma262regex

To be clear, to do this one would likely need to write an entirely new regex engine. ripgrep's default engine uses finite automata. It's not PCRE2 or ECMA. Many of the things in Raku likely can't easily be implemented in finite automata.

ripgrep does have fairly complete UTS#18 Level 1 support already.

> likely need to write an entirely new regex engine

Indeed.

Also, it should be noted that the Raku Regex engine is not a state machine as such. The regex "slang" in Raku, is basically just another way to write code. Code that uses some state machine primitives underneath, for sure, but still code.

A grammar in Raku is nothing other than a class in which the methods are codegenned using the regex syntax. But a class nonetheless, which can also have attributes and "proper" methods.

This gives the Raku regex engine the flexibility needed to be able to parse Raku source code, and generate executable bytecode from that.

forgive me for assumption that rg does PCRE(2)/ECMA, we had a bit of a chat on raku and UTS#18 here ... https://www.reddit.com/r/rakulang/comments/15ge67k/uts_18_un... ... raku does not attmept to implement that explicitly but (by virtue of being an HLL and having built ins such as Set operators) comes quite close to it at all levels.

Nice, thank you for the link!

And yeah, I figured Raku might not follow UTS#18 to the letter of the law with respect to syntax, but likely has all the concepts available. Which makes sense given what Raku is.

Reply on news.ycombinator.com