Comment by jgraham | Hacker News Reader

Comment by jgraham | Hacker News Reader

This seems to fall into the common trap of misunderstanding the causality in standardising legacy features of the web platform. The post says:

> The [WHATWG] spec says so because browsers have implemented the spec.

But that's exactly wrong. The WHATWG spec is based on the observed behaviour of browsers. By documenting what existing browsers do, and scoping out who is prepared to change their implementation when there are differences, it tries to specify a behaviour that is either already implemented or able to be implemented with as few changes as possible. It is possible to specify behaviour that is stricter than any current implementation, but the burden of proof for doing so is non-trivial; you have to demonstrate, to the satisfaction of all involved parties, that the proposed change will not break enough web content to cause a compatibility problem. Doing that is non-trivial, but possible if you are sufficiently motivated.

The complaint that the WHATWG spec is written by a closed group also seems like a misunderstanding. Compared to pay-to-play organisatons like the W3C, or pay-to-meet organisations like IETF, there is no concept of formal membership, or need to pay to get your voice heard. There's a mailing list, an open GitHub repository and an IRC channel. Anyone, especially prospective implementors of the spec, is welcome to participate in those forums, and they will have the same privileges as any other contributor. I imagine that for URL in particular, contributions from the CURL comminity would be most welcome.

It is good to have clarity on how WHATWG is working. Thankyou! Sounds like an HTML5 style treatment of the problem.

I'm not clear though why the effective starting point is the superset of all variants currently implemented by one of the involved parties. Given that the extant standards are quite strict, couldn't an argument be made that the "burden of proof" would be on why these needed to be extended, on a case by case basis?

Regardless of the particular agreements reached, an up-to-date standard will certainly benefit everyone; it seems like it would be an excellent outcome if the standard came with a reference implementation too :-)

Browser vendors will be hesitant to remove any special variant that might break existing websites.

The number of websites with this issue is so small that in my 9 years of crawling the web, I'd never noticed the issue.

Why do you think you would notice it? I imagine the only giveaway would be the url in status bar when you hover mouse over the link. If you are a programmer (or very pedant person) you might notice it, otherwise probably not.

Also, I hope they acted on some real-world statistics, provided by Google's or other crawlers...

I was the CTO of a web-scale search engine, and now I work on the Internet Archive's Wayback Machine. I'm not referring to things I noticed as a web end-user.

It's not exactly the case that it's the superset of all variants currently implemented. In fact that doesn't really make sense; two implementations can just have contradictory behaviour so there is no superset (well, unless you allowed things like picking one of the variants at random, I suppose ;)

In fact the process is iterative. When you find that implementations do different things, you make an educated guess about what's most likely to be compatible with existing content. For example if all browsers allow an arbitrary number of slashes between the scheme and the host part of a URL, it seems very likely that some existing content relies on that, and very unlikely that you are going to get all the browsers to change their implementation, so you standardise that behaviour. On the other hand if you find that, say, one browser allows a semicolon rather than a colon after the scheme, but no other implementation does, it's very unlikely that is required for compatibility, so you typically don't allow colon-after-scheme in the spec, and speak to the existing implementation about fixing their bug.

The hard case is where there isn't a clear consensus about the required behaviour, especially when implementations are flatly contradictory. In that case you speak to people and find out whether they have some evidence that their current behaviour has good effects, and whether they are willing to change. Armed with this, and your ability to reason about what should be more compatible or otherwise desirable, you pick one behaviour and hope that people converge on it. Often, in these cases, time will pass and it will be clear that you made the wrong choice; in that case you go back and edit the spec to reflect whatever reality turned out to be.

So the process is iterative and not perfectly algorithmic; the person doing the work has to apply their judgement to make decisions where there isn't a clear path forward. But the goal, at least, is clear; it's to have a single specification that is in-line with what's actually implemented in the real world, and what's needed to consume real-world content. In this sense the URL spec is already a success; for example it was used as the basis for the Rust url library, which is believed to be compatible enough to consider using not just in Servo, but also in Firefox in place of the legacy C++ parser.

Reply on news.ycombinator.com