Stop avoiding regular expressions damn it
bradt.ca
bradt.ca
The fact is that despite pithy slogans, there is a sweet spot where a regular expression does the job of matching a string in a clearer fashion than anything else. But that sweet spot is well shy of the theoretical power of regular expressions (especially in Perl!), before which you should further your understanding of a range of parsing techniques before hacking together a baroque regex.
A couple of years ago I moved to a different country, and for some reasons I needed two apartments, preferably close to each other. As you can imagine, the real estate websites are not designed for the kind of query I needed, so I wrote some code to aid me in my quest[1].
It's just shell script and text processing with awk. I download various results with all the available apartments for many real estate websites, then I scrape the data I care about (with regular expressions!) like address, rooms, price, anything really, and query the Google Maps API with all the addresses to retrieve the geographical coordinates, then I compute the distances between any two houses and sort them.
It's fantastically modular. Adding support for a new website meant just creating some regular expressions that work for that website. This was great because I was doing this on the road, as I was visiting the foreign city and found new sources of information.
Regular expressions were also great because these websites didn't have any API where I could query for the address, etc. I had to rely on what people wrote in their ads. This meant that when I wrote a regexp to match a set of results I had to inspect the failures to see new ways people described their houses and improved my matching based on that. Initially I had hoped I'd be able to parse 80% of the ads, but measurements and careful coding had allowed me to match approximately 99% of the ads!
The textual operation of this software allowed me to easily input some data manually. For example I realized that I'm also interested in having these apartments close to a subway station. No problem, just manually create the file with the subway stations in the correct, simple, textual format and the program will pick it up and use automatically.
The textual interface also helped with fancy queries, like "price between X and Y, 6 rooms total, prefer 4-2 to 3-3 if distance less than D, but 3-3 if distance greater than D, prefer Z subway line to Q, only one apartment might be from an agency rather than an individual, try to put one in K part of the city". Try to do that with an existing website.
I noticed my coworker was going out of his way to use string manipulation, writing many lines of code instead of a simple regular expression. When I asked why, he explained that he didn't know regular expressions, but more importantly that he felt that he had read a lot of posts on Stack Overflow discouraging use of regular expressions. From what he had read, he felt that it was better practice to avoid regular expressions. Although this could be anecdotal, there may be a real danger here that inexperienced programmers are getting the wrong message, that regular expressions are somehow bad in most situations and not worth learning.
BenchmarkRegexp 500000 5136 ns/op
BenchmarkStrings 10000000 173 ns/op
The problem is when developers that don't know any better build parsers with regular expressions. That's almost always a bad idea.
Realistically, you'd expect them to behave exactly the same, but Go's pretty new, and you never know what is or isn't going to be optimized.
`\A@(.*)@\z`
BenchmarkRegexp 500000 5181 ns/op
BenchmarkStrings 10000000 171 ns/opMustCompile is generally used to make the regexp a global so that it isn't done over and over.
Just move it out of the loop, as it's really not necessary to compile regular expressions every time you want to match/replace against it.
For trivial replacements string manipulation I find is faster and safer (fewer bugs). But there is some threshold of complexity in which regular expressions are both more performant and safer.
The classic example is to recognize that you're looking for a fixed string, and substitute in Boyer Moore. But prefix/suffix recognition are two other common examples.
My current pet peeve is with parser combinators, wich seems a good compromise (it's not a magic wand) between maintenance (whereas external parser generators don't blend well in your code), parsing what you think you are parsing (more so when your grammar was defined with rules in a reference document), and integrating the parser with your code.
I think this would be helpful in many cases in getting people to understand how regexes work. I've seen lots of cases where toolsets designed to help people build regexes end up with them confused when their regex also matches other stuff beyond their test strings.
By the way, if some like spacing that much, and if the reason is to have a better mouse-selectability, then I humbly propose "( a , b )".
Basically a great online tool for testing your regular expressions and stepping through what is actually happening. As soon as you get non-trivial, it's a Godsend.
You will profit from it for the rest of your career.
(There's also a Regex short reference and a Regex cookbook by O'Reilly...)
The book doesn't just teach you regex, but the why, how AND the dialects. It gives you an overview over different tools and programming languages and their regex-related functions and methods.
On top, it contains a ton of examples, is very well written (considering the insanely dry and difficult to typeset subject :) and is very polished (I think it's in the 3rd edition by now..)
If you just google or experiment on regex, you usally get bad regex, badly crafted regex, brittle regex and make every single mistake the book prevents you from doing.
It's really one of the most worthwhile books of reading through - it's also an excellent handbook to look things up.
Remember that a lot of commandline tools take in regex too - grep, sed, awk, you name it - it's not just for use in programming languages.
Your favorite editor has regex too.
I simple don't know how people can live without; I'm using regex practically every day.
P.S.: And _after_ reading the book, you will understand why people yell at you when you parse HTML with regex but you will know how to do it anyways and at least not completely badly. ;)
P.P.S: And here's the canonical post to BUT OF COURSE you can parse HTML with regex from stackoverflow.. :) http://stackoverflow.com/questions/4231382/regular-expressio...
I do take issue with your suggestion that you might not use this stuff all that frequently in your career. This is definitely at odds with my experience. Even though I don't use them that much in final-quality code, I use them all the time from the text editor, and quite often for quick one-off text manipulation or extraction scripts. Having a quick way to extract text from ad-hoc data can quickly get you a rough answer to a speculative question, the text equivalent of of back-of-the-envelope calculation, without needing to do a lot of work and without needing the question to justify a lot of work.
But I mainly use them for searching for one of two or three different strings in the text editor.
An interface that e.g. makes me 'escape' half of my input because its designers think their special use of characters must take precedence over all user input is a bad interface.
In Perl, it's called quotemeta (qw, qq and family, too), in Python and Ruby it's .escape... and there's always \Q ... \E to use...
I'm sure others have similar methods/functions.
That always makes me giggle.
If I recall correctly I learnt the basics from MSDN documentation and later more thoroughly when I first came across Perl. Either of these are pretty decent choices, too. :)
Eg: /^([a-z]*)$/ this translates into my brain to: a string that starts with a capture group of 0-N characters between a-z and also ends with it.
Then it's really easy.