But any regex engine that can work with a parse tree shows the same principle, e.g. https://edicl.github.io/cl-ppcre/#create-scanner2
Since you explicitly talked about filling the fields in an f string with already parsed regex objects instead of strings, it's hard to see what else you could mean. But even if I s/regex engine/DSL parsing engine in general/, I would like to see an actual example of a language or library where I can have a string like a Python f-string whose fields can be filled with some kind of parsed "engine" object instead of another string.
> Composing DSL programs by string concatenation is such a famous source of security bugs you see it in top-10 lists.
I don't see how composing DSL programs by filling in string fields with parsed "engine" objects is much better. I personally don't like regexes in general because I find them too hard to reason about unless they're extremely simple (and regexes that simple usually aren't necessary). I would rather try to write library functions (which might include functions that build other functions) in the same language as the rest of my program.
You could make one called, say, rx for regex. Then
rx`${a}|${b}`
would evaluate to exactly the same result as a tree constructor call like regex_or(a, b)
given corresponding definitions of rx and of regex_or. There's never any question of whether a and b are escaped right. So it brings the composability advantage of the library functions you prefer, to people who want to write these concrete-syntax regular expressions that started the whole thread.I mean python's regex module was added in 1997 and hasn't fundamentally changed since. I don't think that concept was super common in '97.
(E did have this concept back in '97, iirc, though yeah I wasn't expecting Guido to have run into it then, that wasn't when I meant.)
An f-string fills holes in a format string to build a string.
A template literal parses a template with holes to produce any datatype you like, filling it with arguments of any appropriate type.
This does for Javascript what f-strings do for Python (with similar syntax and simplicity), but also more. Besides the greater expressiveness, it can catch bugs, in the same way that Lisp macros are safer than C macros. It's not hardwiring regexes, it's not hardwiring any datatype: it calls a function that you name in the tag, which you can define to do whatever parsing and filling is proper.
Lisp's quasiquotation is similar in spirit though different in appearance.
This is my last try to explain here. I guess this thread shows that tagged template strings are much less well known on HN than I thought. (Yes it also shows me I was unusually bad at communicating.)
> the complaint that you wish the formatting API made regexes a first class citizen is odd
That was not what I was trying to say. The template literal mechanism knows nothing about regexes. Regexes are just one particular type and one particular syntax.
I think the bigger problem is not so much that regexes are generally defined by strings specifying the desired regex, but that at some point between the development of printf and the development of regex libraries people forgot that they could use whatever escape character they wanted when implementing a new conceptual data type. In C, the compiler deals with strings, and the printf function deals with strings, and they try not to conflict with each other by assigning the escape character \ to the compiler while printf uses % instead.
But in Java, and Python, and presumably many, many other languages, some idiot decided that if strings used \ for their escape character, regex functions should also use \ for their escape character. Since they accept strings, suddenly the regex escape character is actually "\\". How do you match a single literal backslash? "\\\\", obviously. What's wrong with that?