Getting started with regular expressions: An example
redhat.com
redhat.com
Regardless, I love this tool: https://regex101.com/
I usually hammer out a prototype there before testing in any method if I have to write anything non-trivial or that may have edge cases I want to test out.
I've had amazing success in using a regex in situations where you wouldn't think it would work as well as other solutions. For example, I've gotten more than an order of magnitude speedup in parsing a well known simple (but fairly large) XML data set using regular expressions instead of the fastest XML parsing libraries I could find at the time. Sometimes the less efficient tool that does only what you need is much faster than the highly optimized tool that handles all the special cases that don't matter for the particular job.
<doc>
<bool name"foo">1</bool>
<int name="bar">12345</int>
<str name="baz">test string</str>
<float name="quux">1.234</float>
</doc><doc>
<bool name"foo">0</bool>
...
</doc>
my @items;
my $count = 0;
while ($item_xml =~ m{<doc>(.*?)</doc>}gsmi) {
try {
# process <doc>
my $item = $1;
my $i = {};
$i->{$1} = $2 while $item =~ m{<(?:arr|date|str|bool|int|float) name="([^"]+)">([^<]+)</[^>]+>}gsmi;
push(@items, $i);
$count++;
}
catch {
warn "Error :: $_";
}
}
print "$count items found\n";
my $field_parse_re = q{
# Parse opening tag
< # Start of opening tag
# Any of the allowed tag types
(?: arr
| date
| str
| bool
| int
| float
)
\s # space between tag type and name attribute
name="([^"]+)" # Save tag name as $1, or first returned item
> # End of opening tag
# Parse tag contents
([^<]+) # One or more characters that are not <, in $2, or second item returned
# Parse enough of the closing tag to make sure we got all the contens
</
}smix;
# Now use it
$t->{$1} = $2 while $ticket =~ m{$field_parse_re}gsmi;
They ignored whitespace (and allowed them to be written over multiple lines, with comments) which made them more readable and allowed referencing regexps (stored in variables) directly within a regexp. I think they also made some of the syntax nicer (eg non capturing groups or lookaheads), and added notation for common things (e.g. like X* except everything is separated by commas) and long names for things like character classes.
This tends to cause confusion, makes escaping complex, and is just really different from most other ways of expressing logic in programming.
Even advanced regex features like extended mode and named captures, which I love, don't get around this fundamental issue with the representation.
It does have advantages though, a big one being terseness, and the ability to be expressed in an implementation-independent way. Another is that for some types of regexes, the expression "sort of looks like" the strings it can match against, which can help intuition.
I think the way forward is to consider the existing syntax a "terse mode" and to use regex builder libraries [0] as a "verbose mode." The "terse mode" does have merits, though, and I wouldn't want to get rid of it or anything.
[0]: For example https://github.com/VerbalExpressions/JSVerbalExpressions is a library that lets developers express regexps using a fluent API. This changes things from being syntactic elements of a string to just being traditional function calls.
Except from what I can see, it only supports trivial cases. What's a syntax that supports grouping, alternation, a numerical range of number of matches (e.g. \d{3,10}), and the combining of any number of those those? I suspect it quickly devolves to the case where you either use the extended syntax and comments are really what matters, or the terseness is useful in at least not making it so verbose and meandering that it's hard to comprehend just because the amount of boilerplate to usefully express something dwarfs the functional bits.
Regular expressions work best when matching a a character level. The non-terse mode of stuff you are grasping at works best when matching at a symbolic level, and has existing for a very long time. They're called grammars. A popular form is BNF[1] (and variants), which will probably look somewhat familiar from technical specifications or RFCs, if you've looked at any before.
If you want something beyond regular expressions and you're willing to use a library for it, just use a library that provides grammar parsing. Or use Raku (fka Perl 6), which has support built in (with hooks for calling code at particular parsing points).[2] Hopefully most languages have a good library for parsing grammars at this point.
[1] http://www.snobol4.org/docs/burks/tutorial/ch4.htm [2] https://github.com/spitbol/x64
* The global or g switch. The global switch applies a pattern to all matches in a string, which can be used to return all pattern matches or apply all pattern changes.
* In JavaScript space like characters are represented by the \s switch. There are many characters classified as white space, so it is very nice to have a grouping for them when performing a search and being able to ignore them all if necessary. \s+ will match a group of consecutive characters that may be any combination of space, new line, carriage return, form feed, and many other characters.
* Partial matches are helpful. Sometimes you know the pattern you want to search for and you need to replace that pattern with a different pattern without harm specific data in the match. In this scenario I perform a replace method on a string and search by regular expression with the a g global switch. For the replacement result I supply a function name. A regular expression match is a string argument in that function. In the function I can manipulate that result to be something else, even using regular expressions, and return the result, which is inserted back into the target string over the regular expression match.