> likely need to write an entirely new regex engine

Indeed.

Also, it should be noted that the Raku Regex engine is not a state machine as such. The regex "slang" in Raku, is basically just another way to write code. Code that uses some state machine primitives underneath, for sure, but still code.

A grammar in Raku is nothing other than a class in which the methods are codegenned using the regex syntax. But a class nonetheless, which can also have attributes and "proper" methods.

This gives the Raku regex engine the flexibility needed to be able to parse Raku source code, and generate executable bytecode from that.