> Another practical usage I've heard: match "string" or 'string', but not "string'.
You don't need backreferences for that:
'[^']*'|"[^"]*"You don't need backreferences for that:
'[^']*'|"[^"]*"They are fairly powerful in terms of what they're capable of parsing however (not enough for an arbitrary html document, but enough to handle the hairier situations in this thread that people thought they couldn't), and that does mean that a regular expression generator can handle all of those situations as well and potentially be much more readable.
If I found myself writing code like this I'd still want to reach for a better parsing technology, but you can use other languages to add abstractions to regex. Here's a Python3.6+ example assuming any desired backslashes have already been applied:
'|'.join(rf'{a}[^{b}]*{b}' for a,b in pairs) /"([^"\\]|\\.)*"/
What you can't match without backreferences, however, is strings with customisable terminators, e.g. the behaviour in sed that whatever character you use after `s` is the regex terminator (it doesn't have to be `/`), or raw strings in C++.
> Won't work if you're already in a string
This doesn't make sense. How can you search for a string if you're already in a string? I can't think of a realistic situation where that would be useful or even really possible.
> or if there are escaped quotes in the string.
Solvable:
'(\'|\\|[^\'])*'|"(\"|\\|[^\"])*"
The regex in my previous comment already solves that. See: https://repl.it/repls/SolidCapitalProgram
query = "select * from table where name like \"%foo\""Although if you want to do it with pure regex, it can be done without backreferences too, although it would be exponentially large as you get more and more levels of nesting, whereas with backreferences I think it would only get quadratically large.
But more broadly, any situation where you search from a non-zero index has this problem.
I'm surprised your example works in Python. Is that a property of Python's parser, or all regex matchers?
Yeah, I agree. My point was that regex won't work, regardless of if you have backreferences or not. So backreferences won't help.
> But more broadly, any situation where you search from a non-zero index has this problem.
I'm not sure I understand that. A lot of regex libraries let you specify a start index. It won't take into account data from before the start index though (regex doesn't really do that, regardless of backreferences). If your regex library doesn't support passing in a start index, you can just take a substring starting at that index, then search the substring.
I don't think Python is really special. Python's findall() is just a convenience function that does a loop finding a match, then finding another match that starts after the first match, etc. Most languages provide a way to find the end point of the most recent match, and then you can just write the loop yourself to start the next search at that point.
"abc{ "def" }"
"a{ "b{ "c{ "d{ "e{ "f" }g" }h" }i" }j" }k"
→ "abcdefghijk"
my $count = 3;
"I went to $count place{ "s" if $count ≠ 1 } today"And that example reminds me that Bash can do something similar:
echo "$(echo "$(echo "$(echo "hi")")")"Basically you can use backreferences for that if you also allow the regex to be recursive.
my $regex = /
:ratchet
$<q> = (<["']>) # the beginning quote
{}:my $q = ~$<q>; # put it into a more normal lexical var
# capture between " and {
$<l> = ( [ <!before $q> <-[{}]> ]* )
[
[
:sigspace
｢{｣
<self=&?BLOCK>? # recurse
｢}｣
]
{$q = ~$<q>}
# capture between } and "
$<r> = ( [ <!before $q> <-[{}]> ]* )
]?
"$q" # match the end quote
# pass the combined string parts upwards
{ make ($<l> // '') ~ ($<self>.ast // '') ~ ($<r> // '') }
/;
｢'a{ "b{ "c{ "d{ 'e{ "f" }g' }h" }i" }j" }k'｣ ~~ /^ <r=$regex> $ { make $<r>.ast }/;
say $/.ast;
# abcdefghijk
`<foo=bar>` is a way to call `bar`, but also save it under the name of `foo`. `$<foo> = …` is a way to capture `…` and save it under the name of `foo`.
---
It is a lot nicer and modular when you use regexes as part of a grammar:
# use Grammar::Tracer;
grammar String::Grammar {
token TOP { <strings> }
rule strings {
# at least one string
# if there are more than one they are separated by ~
<string> + % ｢~｣
}
token string {
$<q> = <["']>
# set a dynamic variable to the quote character
{}:my $*quote = ~$<q>;
<string-part>*
"$<q>"
}
# multiple tokens that act like one
# which is nicer than using |
proto token string-part {*}
multi token string-part:<non> {
[ <-[{}]> <!after $*quote> ]+
}
multi token string-part:<block> {
<block>
}
rule block {
｢{｣ ~ ｢}｣ <strings>?
}
}
class String::Actions {
method TOP ($/) { make $<strings>.ast }
method strings ($/) { make [~] @<string>».ast }
method string ($/) { make [~] @<string-part>».ast }
method block ($/) { make $<strings>.ast }
method string-part:<non> ($/) { make ~$/ }
method string-part:<block> ($/) { make $<block>.ast }
}
say String::Grammar.parse(
｢"a{ "b{ "c{ "d{ "e{ "f" }g" ~ "zz" }h" }i" }j" }k"｣,
:actions( String::Actions ),
).ast;
# abcdefgzzhijk
Every instance of `<foo>` is basically a method call.
`make` is about generating an `.ast` to pass up and out of the parse. In this case the only thing the actions class does is return what would be the resulting string if it were compiled in Raku.