Tragedy of the WebKit Commons | Hacker News Reader

Tragedy of the WebKit Commons

blog.methvin.com

120 pointsdmethvin57 comments

One problem I've run into is not being able to submit bugs because my own codebase is relatively huge.[0]

For whatever reason, the bugs I see manifest as edge cases in Chrome's new compositing system, but only under "load". I've tried to get small examples that show the bugs, and failed.

But since the developers will only accept tiny programs that demonstrate the problem, I'm SOL.

UPDATE: Also, there's no such thing as "WebKit", as though rendering issues, once fixed in "WebKit" are suddenly fixed as long as everyone updates to the last release. It doesn't work like that, because there are numerous rendering implementations built-on top of WebKit -- the ones I run into have to do with Chrome's compositor vs. Apple's compositor. They behave quite a bit differently, and those aren't the only two in use. The same is true of WebKit JavaScript engines and the APIs supported.

There's just no way, as a developer, to treat "WebKit" as a platform. You have to deal with every browser using WebKit separately.

[0] https://github.com/erichocean/blossom

> I've tried to get small examples that show the bugs, and failed.

Have you tried setting up an automated process which tries to gradually reduce your large codebase to a small one which still reproduces the bug?

Some similar existing projects: Tigris Delta for C/C++ code, Bugpoint for LLVM, and my own DustMite (https://github.com/CyberShadow/DustMite) for D code. I wonder if something like that already exists for HTML.

I haven't, although I'm sure with enough effort, I could build a one-off tool to figure out what exactly the problem is.

If I was developing Chrome's compositor, sure, I'd do that.

One difficulty is that the "bad" code is actually generated HTML and CSS. Well, the generated code is fine, it's the browser that eventually has problems with it, but you get what I mean.

There's probably a way to capture that output in buffers over multiple event loops, and then do something like Bugpoint over that output and see if I can't reproduce/reduce the issue.

I'm also not sure if the Canvas drawing I'm doing is part of the issue or not, so I'd probably have to capture that as well...

Do it using a dichotomic approach. Remove 50% of the CSS. If the bug is still there do it again. Etc. Do the same for HTML and javascript and you will have your minimal test case.

You're assuming that the issue isn't actually caused by load, even though he said it is.

It's really interesting to get the opinion of a jQuery core contributor on this whole thing. I can't imagine very many people are in a better position to comment on the state of current WebKit bugs than the folks writing the most popular DOM/AJAX library on the web.

He had me until he put bloated (as a description of jQuery) in quotes.

Admittedly, he was refering to the 2.0 line, the goal of which was to drastically lower the size of the codebase by cutting out support for non-current browsers.

Feature bloat is still bloat. jQuery is a DOM library, an XHR library, an animations library, a functional library, a promises library... as far as I know 2.0 doesn't do anything about that issue.

In either 1.9 or 2.0 you have the option to exclude XHR or animations as well as several other functional modules. More feature-lopping capability will be coming soon.

> Even when they have been fixed in the latest Chrome or Safari, older WebKit implementations like PhantomJS and UIWebView still don't have the fix. We've had to put back several of these as users reported problems with the beta. It's starting to feel like oldIE all over again, but with a different set of excuses for why nothing can be fixed.

I am fairly new to web development, but this consistent droning (whining) about having to develop for a myriad of inconsistent platforms seems like braying from rather entitled web developers. I don't understand when in the history of software development developers weren't forced to dick with platform and legacy problems. Can someone explain what, if anything makes web dev different? And if they can't, can we please just stop bringing it up like it's ever been a new problem?

Is the "whining" particular to web developers either, or have developers always complained about the platform and legacy problems they've been forced to dick with (look at the angst over Android fragmentation for example)?

Before getting into web dev I spent a lot of time doing development in C/C++. That code needed to be portable across systems and versions but it was an entirely different kind of portable than we grapple with on the web. For C code we could handle the issue with conditional compilation, build scripts, or dynamic loading.

On the web it's all dynamic; all the code for handling every situation you might encounter comes down to the client when the page loads and you sort out the details at runtime while the user taps their fingers on the table. If there is a large amount of code to fix one specific browser you can break that out and pull it down only for that browser, but the kind of bugs that jQuery works around tend to be of the death-by-a-thousand-little-cuts variety where it just doesn't pay to refactor that way.

He touched on something that few will appreciate.

UIWebView, (and android's webview) is on older implementations of webkit, and the gatekeepers of modern apps (Google included) have incentive to keep it that way.

They want devs to build apps for their own 'ecosystem' that are not interoperable on all platforms (eg: the web). They have disincentive to push for adoption of newer versions of browser tech as it encroaches on their respective babies (the OS').

No it's not. Both you and the original article author are astoundingly wrong here (with regards to UIWebView; I don't know anything about android). I don't even know where you got the idea that it's somehow older.

UIWebView is precisely the same WebKit that powers MobileSafari, which is typically kept in sync with WebKit for the corresponding OS X release (so iOS 6.0 and Mac OS X 10.8 have roughly equivalent WebKits).

Pretty much the only difference between UIWebView and MobileSafari is the fact that MobileSafari runs JavaScript faster (by virtue of being allowed to mprotect pages, which almost no other app on the system, including 3rd-party apps, are allowed to do). But aside from a speed difference in JS execution, UIWebView is the same WebKit that's used in MobileSafari.

Mobile Safari is only updated once a year.

Nope, it's usually updated with every single minor and security release as well.

Only receives new features once a year. Safari (mobile or desktop) isn't exactly a model for rapid releases. It's missing a lot of stuff that Chrome has had for months, sometimes years.

Which is a very different thing to say than "UIWebView is an older implementation". Especially when you're talking about removing something from jQuery that was meant for older versions of WebKit only to have to put it back when you discover that, shockingly, there exist browsers out there that aren't on the bleeding edge of WebKit!

Not everybody runs Chrome.

I'm not sure what you mean by UIWebView being an "older implementation of webkit". UIWebView uses the same version of WebKit as Safari on iOS, which is essentially the same version of WebKit as is shipped with Safari on OS X.

Right, the difference is Mobile Safari and home screen web apps use the Nitro engine while UIWebView still uses JavaScriptCore IIRC.

Nitro is Apple's marketing name for JavaScriptCore, so UIWebView and Safari both make use of Nitro. The difference you're thinking of is whether a JIT compiler is used. That difference has nothing to do with the age of the version of WebKit used in UIWebView vs Safari.

For android webkit versions I'm using: http://jimbergman.net/webkit-version-in-android-version/

and iOS and the rest: http://www.quirksmode.org/webkit_mobile.html

and good luck actually finding a sold version history of webkit on iOS, 30 minutes of googling have led me nowhere.

Sounds like jQuery could benefit from committing their own fixes to WebKit rather than working around them. If the bug Opera fixed really was just one line then why did they have to wait for Opera to do it?

That doesn't stop their need to work around the bugs in the existing version. It's easier for the jQuery maintainers to work around a Webkit bug than to submit a patch and hope that it gets applied in the next few years.

EDIT: I misread the article on how the bug was actually handled, and that it was fixed five years after it was reported, not that it took five years to apply the patch. However, it still is easier for the library maintainers to use a workaround than to patch the browser itself, especially when legacy versions still need to be supported.

"submit a patch and hope that it gets applied in the next few years."

What are you talking about?

Comment #5 From Morten Stenshorne 2013-02-13 00:01:06 PST Created an attachment (id=188026)

12 hours and 12 minutes later:

Comment #11 From WebKit Review Bot 2013-02-13 12:13:22 PST Clearing flags on attachment: 188026

If you read the linked post the complaint is that vendors like Apple and the developers of packages like PhantomJS regularly ship versions of WebKit that are months or years out of date. the jQuery dev team could fix every outstanding bug in WebKit and they would still have to ship compatibility hacks in jQuery for years-old builds of WebKit.

If you read the linked post the point is that if they fixed every outstanding bug via a JavaScript workaround, everybody else would have even less incentive to fix anything.

Also, if there is a long delay in the pipeline, that is no justification for even more delay in putting something into it, just like things being put into it quickly would be no justification for not trying to shorten that delay.

If you don't fix any bugs, but Apple etc. update overnight everything via WiFi or Ninjas, jQuery will still keep accumulating fixes for old bugs and explode - on the other hand, if you fix bugs quickly, even a 3 year delay with vendors means that jQuery will "only" have to carry fixes for bugs of the last 4-5 years, not for bugs that existed since the dawn of time.

I think that is not even the job of jQuery, anyway; it fixes bugs that have to do with using jQuery and plugins, providing a consistent baseline despite browser bugs.

I suspect that was a well-orchestrated commit to show that Opera had joined the WebKit collective. It was mentioned in Opera's press release.

There are a lot of really talented people working on jQuery, but I'm not sure how many of them have much expertise in fixing rendering engine bugs.

1. Just because a bug is fixed and accepted doesn't mean it will make it into a shipping browser anytime soon.

2. Even if it were to make it into a shipping browser quickly, there is no guarantee any reasonable amount of people will run that upgrade anytime soon.

Hitting browser bugs really sucks for this reason. The bug has to be worked around no matter what, since there will probably be people running that bug for a long time (this has gotten much better with things like Chrome's auto-update which is why if you talk to people from that team they are such big believers in it). Regardless, you have to remember that the jquery team is providing a product to clients. If a client's site is broken because jquery hits a bug in a browser, the priority is to have that site work asap. Its unacceptable to say "Oh we fixed the bug in webkit, Safari users will see it whenever the new Safari ships which we have no idea when that is since its an apple secret. Bug closed." The best response is "Fixed in new jquery version whatever, just update the script on your site". And once you've fixed the bug for your own concerns, it just doesn't make sense to then spend time fixing it in WebKit as well vs. spending those resources on fixing another jquery bug.

So, moral of the story is don't fix bugs because it might take a long time to make to the end user?

No, it's that even after bugs are fixed in widely-used libraries, they will still exist in downstream projects and in the wild for quite some time. Given that jQuery exists to paper over implementation quirks (including upstream-fixed bugs) between real-world browsers, jQuery needs to worry these bugs long after they are fixed in the core library code.

There is no moral, its just an observation of realities. From these circumstances judgement calls are made on a case by case basis. Maybe someday when all browsers have auto-update like Chrome it will absolutely make sense for everyone to focus their efforts on fixing bugs in the engine vs. writing workarounds. That is not the reality we live in today.

I mean, I could argue that people shouldn't work around iOS bugs and should instead apply for jobs at Apple and fix UIKit itself so that everyone can best benefit from their labor. But that's absolutely absurd. It's also more than a little unfair to judge members of the jquery team for not going into a project like WebKit that's millions of lines of C++ and acting like they're coming up with excuses for "not fixing bugs". This is a symbiotic and healthy relationship, browser vendors benefit from groups like jquery that make those bugs livable today through workarounds so that their products can be more usable in the immediate and they can focus on the "real" fixes for the future.

And don't forget that there are people who are running old OSes that would require buying a new computer before being able to upgrade their browser.

That's perfectly realistic, but it's the same 'tragedy of the commons' that the jquery developer is bemoaning in browsers. They stand to benefit (eventually) from fixing Webkit and improving things for everyone. But in the short term, their time is better spent focussing on their own project, and hoping someone else fixes Webkit.

That wasn't a bug that jQuery reported or has encountered. The ones we report probably aren't half a line of code to fix. And as for why we don't fix them ourselves, I can't speak for the rest of the jQuery team but one unpaid volunteer open source job is plenty for me.

Is it safe to assume that any bugs that you're running in to are being reported to the browser vendors?

The most telling thing that jQuery (or any other library) could do would be to have a policy that any browser bug workaround has to be accompanied by both a) a bug report to the browser vendor and b) a corresponding test to the relevant W3C testsuite. That would put public pressure on browser vendors to fix the issue, and provide the jQuery team with a convenient way of telling when the workaround was no longer needed.

Speaking as someone that has spent time working on a web browser, that would be a fantastic policy, both for browser developers and for web standards in general.

Sadly, from what I can tell, at least some browser vendors (and more importantly the developers working for them) don't particularly care about whether they pass W3C test suites....

Yes, we try to put a link into our tracker with the ticket in their tracker for both bugs and spec ambiguities. Note that several are a few years old and are still open.

http://bugs.jquery.com/ticket/11663 http://bugs.jquery.com/ticket/12986 http://bugs.jquery.com/ticket/12968 http://bugs.jquery.com/ticket/12964 http://bugs.jquery.com/ticket/10509 http://bugs.jquery.com/ticket/12577

I find the negative tone of this article very odd.

He is (rightly) upset that bug fixes are not glamorous, and not submitted as often as they should be to WebKit.

Yet the fact that Opera has moved their team onto an open source project, and is actively submitting bug fixes at all, is surely a positive thing?

The alternative is they continue to develop the Presto rendering engine, to which no one but Opera can submit bug fixes, and this bug fix wouldn't have been submitted to WebKit at all.

So surely this move is a positive gain from any perspective? I'm not sure why the negative spin.

I think he is arguing that most browser vendors don't fix common bugs as much as they add features.

I understand that point, and agree. Just seems odd to use the news of Opera adopting WebKit as a platform for complaining about vendors not fixing WebKit bugs.

As far as I can see, Opera's adoption of WebKit can only result in more bug fixes submitted for the framework.

He does make a fair point that browsers will focus on themselves and not on fixing common bugs, so in the end benefit to WebKit will be less than anticipated by such move.

Also he does compare WebKit to oldIE, though I wonder is Gecko any better.

I was only comparing the version-spread problem of WebKit to the one we have in IE, not comparing the two browsers.

My bad. I guess Gecko isn't prolific enough to have major difference between versions/forks.

No, it could even mean fewer fixes, because coopetition is a delicate balance. Every man-hour you put into the common project helps your competition as much as yourself (outside competition like Mozilla and Microsoft excluded).

So if you remove some coders from the common codebase and shift them towards other parts of the browser that have more to do with your specific brand (JSC in Apple's example, V8 or cloud integration in Google's), up to a certain point you gain a net benefit - your brand develops faster and your partners invest more.

Of course, if each party acts like that, the common project gets stale - the tragedy of the commons.

From my completely unfair outsider view, it seems to me that Apple, for example, has more or less lost interest in WebKit and tries to stifle progress to defend their lackluster Safari against Chrome and MobileChrome.

Do you have specific examples of how you consider Apple to be "stifling" progress to defend Safari?

I use both Safari and Chrome (and WebKit nightlies). I vastly prefer Safari to Chrome, but their underlying use of WebKit seems quite similar.

Competition has seemed to help WebKit in the past (especially between Apple and Google). If you recall, they both contributed a new process model to WebKit (http://betanews.com/2010/04/09/the-big-change-coming-to-safa...). It could be argued that Apple's contribution in this area was more useful to users of the WebKit framework, while Google's was restricted to Chromium. That said, Google makes many great contributions too.

I love how this post is the total opposite of John Resig's article about how WebKit is the jQuery of rendering engines: http://ejohn.org/blog/webkit-is-the-jquery-of-browser-engine...

And how the "bloat" added by adding WebKit specific hacks to jQuery is just 3 half-lines of code, I think it would be safe to assume that if Opera was as big as WebKit is jQuery would also need a lot of hacks for it.

This is indicative of a systemic problem with end-user software in general, not just browsers. In my experience in the audio world, the DAW-maker's rush makers to crank out features to the neglect of the functionality of their product has been simply horrific. I've used 4 of the top programs (ProTools, Sonar, Cubase, Logic) in paid sessions, and new releases of each one always leave old bugs unfixed as well as introduce new bugs, some of which are crippling. Each one of these programs cost over $500 (at the time they were used, Logic is less now, I believe but also the buggiest and most unstable.) The author is right that fixing bugs doesn't get headlines, but the neglect exhibited by many in the software industry is downright shameful.

>at the time they were used, Logic is less now, I believe but also the buggiest and most unstable.

Citation needed. Cubase has horrible bugs. And Logic 9 for example HAD 9 service releases, with bugfixes and new features IIRC.

Unless you're Apple.

I was really shocked when one of the main features of the latest version of Adobe Illustrator was "we went in and rewrote a bunch of old messy core programming and made it lots faster under heavy loads".

Of course, they also rebuilt the whole UI from scratch and introduced some new bugs in that...

The way this is going WebKit is the new IE.

Blogger like to refer to WebKit as if every browser in this planet would be using the latest version, which sadly is not the case.

So one ends up coding multiple hacks to support several versions of WebKit.

Even with help of jQuery, there are CSS and HTML rendering issues to work around.

I don't really understand the difference between WebKit and Chromium. Is Chromium the browser implementation using WebKit?

I reported a bug to Chromium last year which was to do with using screen.availHeight in Windows. The value returned ignored the taskbar which made this setting unreliable. Using it to open new Windows meant the Window would fall behind the Taskbar.

Its still bugged. Since then people have periodically confirmed it. It doesn't surprise me that WebKit has the same problem with edge case's being ignored in the rush to add more functionality.

This does actually make me want to get in and fix some of things. Perhaps if some free time opens up I could make a difference here.

If you are going to reference "Tragedy of the Commons", do it correctly by pointing out what a horribly simpleminded fantasy the whole fiction is.

EDIT: for those that care, the original paper by Garrett Hardin simply explored the idea. He did not use real data and it was not scientific in any way. It was a thought experiment. And generally proven wrong.

Why it it relevant to mention this? Because the article mentioned here about Webkit references it in some attempt to connect it to the idea that Open Source or Free Software is an example of Garrett's argument.

However, if anything, Webkit is a counter example in an interesting way. One typical example of given to TOC is global fishing. However, if anything, global fishing is an example of large corporations exploiting a resource owned by nobody (vs a resourced owned in common). In a similar fashion, WebKit might be an example of corporations exploiting code owned by nobody.

Incidentally, this is not the case for many Free Software projects (vs Open Source). Because Free Software projects ARE owned by a group of people in common. And typically the governing structure is less exploitative, and more cooperative than corporate controlled projects like Webkit.

Reply on news.ycombinator.com