Hacker News Reader

Comment by xurukefi | Hacker News Reader

In my experience, a regex can be a much more robust solution than an XML parser depending on the use case. Back in the days when I did some webscraping I often had parsers throwing exceptions becacuse of ivalid HTML. More often than not I switched to regular expressions, which always worked out flawlessly.

View on HN

Yes. HTML isn't XML.

As you discovered, HTML in the real world is allowed to be malformed in various ways, while XML is not. A compliant parser MUST barf on various kinds of malformed text. (Search for "fatal error" in https://www.w3.org/TR/REC-xml/ to verify.) This makes XML parsers inappropriate for HTML parsing.

Interestingly, even perfectly valid HTML may not be valid XML. To see that consider <b>this <i>example</b> carefully</i>.

"always flawless" is a bold claim

"I've never lost playing Russian Roulette"

It's hard to imagine any web scraping being described as "flawless". But there's a whole lot of room for "worked fine for my purposes."

I think that it's actually a pretty great example of a case where capturing data from HTML may not be best modeled as a parsing task. You might not need a whole parse tree just to match some pattern and grab an associated string. And skipping the parse may enable you to get useful data out of a file that technically can't be parsed due to syntactic errors. It's a fairly classic precision/recall tradeoff situation.

Reply on news.ycombinator.com