Hacker News Reader

Comment by w0mbat | Hacker News Reader

Ex-browser dev here. Please don't ship invalid HTML like this.

Web standards specify how to render valid HTML in a standard way, and browsers have become better and better at that over the years.

Invalid HTML destroys all that. Each browser will have to guess how to repair and fill in the blanks on malformed incomplete tag soup, and there is no one right way to do that. Browser X will make different guesses from Browser Y, and the next version of each will be different too.

Please just write actual HTML that is valid and your website will render much more consistently and reliably across shifting platforms, browsers and versions. It is not the size of HTML that slows the web down anyway.

View on HN

As of HTML5, web standards specify how to render invalid HTML as well, precisely to avoid this problem:

https://html.spec.whatwg.org/multipage/parsing.html#parse-er...

In addition, most of the suggestions in the link (leaving off <html> and <head>, leaving off quotes for attributes, not closing <p>) are valid HTML in the first place.

To add to this, one of suggestions in the article was to drop the doctype. While that puts the browser in tag soup parsing mode rather than html5, in practice browsers don't actually start diverging much in behavior until you get to truly messed up markup, like malformed tags. So even the more egregious suggestions here are probably still fine, provided that you have equally egregious CSS hacks to get around ancient box model quirks

True, and the suggestions in the article are pretty tame. But let’s set aside the article and talk about invalid HTML for a minute.

I write scrapers a lot (not the irresponsible kind and never for monetary gains) and invalid HTML, while technically parseable and to spec, are often a pain in the ass. You have to bring in a full blown HTML5 parser, and they could be way slower (e.g. lxml.html vs html5lib). Depending on your language of choice there might not even be a to-spec HTML5 parser available.

So, just close your damn tags (except self-closing ones), and close them in order, it’s not hard, the size increase is minimal, it will help with your own sanity and people will thank you for it.

For many sites this is an endorsement not to close tags.

It’s an annoyance, not a meaningful hindrance.

There are a lot more web sites than scrapers though. Why push the work to all the web sites in the world instead of fixing your scraper?

But in any case, you are never getting all the web sites in the world to validate, so what does it matter if you can get some of them to validate? You still have to handle the rest.

"it will help with your own sanity"

Writing unclosed/out-of-order tag soup is extra work, not less work.

What a weird hill to die on.

What people don't get about the web is that there is a significant (and growing) number of pages written by people who are now dead. So all the well-intentioned ideas about "cleaning up" html - it's not going to happen.

It's not a question about being pro or con closing tags or which is "best". The question is about the billions of pages which already exist and is not going to change.

If you write a web scraper, that is just the reality you work with.

Where did I call for people to clean up existing HTML? Nowhere.

Did I ask people to not follow bad advice (intentionally not closing non-self-closing tags just for the hell of it)? Absolutely.

I extremely strongly disagree with your points and conclusion here: why should I, a web author, care because you’re using a slow language that doesn’t have a real HTML parser available? You want to parse my HTML, use an HTML parser.

Parsing an HTML document without a correct HTML parser is just wrong. (I decline to call it an HTML 5 parser, because it’s not strictly that any more, and this stuff is from more than ten years ago: get with the program!) I’d lump it in with using regular expressions to parse HTML: useful in certain situations, but not wise for the general case.

So you have a fast but wrong HTML parser? Find a correct HTML parser. If lxml.html doesn’t parse correctly (I don’t know whether it does or not, I haven’t checked—but suppose it doesn’t), then I maintain that you should never under any circumstances use it in new code. It’s an artefact of twelve years ago, and it’s bad.

So you have a correct but slow HTML parser, and you’re not happy with that? I could say “if you care about performance why are you using Python anyway”, but that would be disingenuous—though some of those sentiments can still reasonably apply. Anyway, your remedy is to find a fast and correct one. I confess myself a tad surprised to find no stable or featureful bindings to html5ever, which is one of the best things along these lines. https://github.com/SimonSapin/html5ever-python and https://pypi.org/project/htmlpyever/ are two things that at least start on this, but it looks like nothing interesting’s happened in this space for about three years. Huh.

But anyway: I decline to adjust my authoring practices because you refuse to use the right tools. Enough other people use the right tools that I don’t need to care. It’s rather like the XHTML/HTML situation if you squint: the only reason to use XHTML, which would reject an invalid document (while HTML would let your nominally invalid document do something useful), would be if something you were interacting with required it.

If your authoring practice is producing wrongly closed tag soup that happens to work (making sure it happens to work takes longer than writing it correctly in the first place), making it unnecessarily harder for everyone including you and your coworkers, then you simply suck as a developer. Of course I'm not paying you, so whatever.

That's the thing, it's not "wrongly closed". XHTML lost that fight.

> As of HTML5, web standards specify how to render invalid HTML as well

Ah, but this post is (seemingly accidentally) recommending you use quirks mode instead of HTML5:

> * <!DOCTYPE html> is required by the HTML spec (but not the XHTML spec). Most of browsers will not care if you omit it. It was created for backwards compatibility with prior HTML versions. You can omit it in minimal documents.

All of the examples provided are valid HTML5, and all HTML5-compliant user agents are required to implement the HTML5 parsing algorithm, which constructs a DOM tree in a deterministic manner for any given input.

HTML5 specifically permits tag omission in many cases [1], including but not limited to both the opening and closing <html>, <head>, and <body> tags. The parsing algorithm provided in the standard generates a DOM tree that includes the omitted tags in the correct place(s).

For example, the following document is valid HTML5:

  <!DOCTYPE html>
  <title>Hello</title>
  <p>Welcome to this example.

[1] https://www.w3.org/TR/html52/syntax.html#optional-tags

The original post specifically mentions dropping the doctype which is not valid. It also implies that you can drop all (or at least many) closing tags without saying that some are optional and some are not.

would you need the doctype if the user-agent said it was html5 compliant?

The doctype is required for it to be a valid html5 document.

Yes. Especially then. Without a doctype, it's not html5.

Without doctype, browsers will render your page in quirks mode, leaving you scratching your head because of margins being set out of nowhere.

https://developer.mozilla.org/en-US/docs/Mozilla/Mozilla_qui...

I’m curious, when were you last a browser dev? Because as noted by other responders, what you’ve said is just wrong (quite apart from the fact that this is all valid HTML, except for removing the doctype), and has been for quite a few years now. Fifteen years ago, yeah, your advice was sound, but it’s been completely safe to depend on an HTML5 parser being in use for most of the last decade.

(I’m not trying to be hostile, just seeking that misinformation be corrected, and you’ve made a claim that doesn’t make sense to me unless it’s from quite some years ago.)

Nothing of this is invalid HTML. It has always been part of the HTML DTD that certain tags are optional and certain tags does not need to be explicitly closed. This is a carry-over from SGML. The quoting rules for attributes are also well-defined, there is no guesswork or error-recovery involved.

Now it this was about XHTML you would have a point.

People often mix-up optional syntax and error recovery. You can leave out the <head>-tags, the parser can infer it - this is well-defined by the DTD. It is not an example of error recovery.

Reply on news.ycombinator.com