Comment by sigil | Hacker News Reader

Comment by sigil | Hacker News Reader

Wow. Good luck there. For the last year or so I've had the distinct pleasure of patching and building WebKit for a project. Some observations about the WebKit codebase...

First off, man is it huge. There's a link stage that uses ~4GB of memory. The command line for that link is so long I needed a patched version of make [1]. Why the WebKit devs haven't knuckled under the pain and broken out multiple smaller libraries is beyond me. They must be tough.

Second off, if you dive into the code -- say you're looking to change something about the layout algorithm -- you quickly discover that there is no layout algorithm. Or at least, no one place that articulates the algorithm, which is instead spread like pixie dust over dozens of objects. Each one contributes a little bit of the logic. (You're in a twisty maze of classes, all alike...)

Perhaps some of this is a reasonable separation of concerns that helps the whole thing scale. But it made me step back and wonder if we're approaching these kinds of problems right, architecturally. Does WebKit really need so much code? If we could go back and write a browser in a functional language, what would that look like? Would performance necessarily doom it from the start?

[1] http://blogs.gnome.org/diegoe/2012/09/23/webkitgtk-failing-t...

The link stage is pretty huge. It currently won't compile with emscripten with debug symbols, in-fact the entire obj-pre-link stage is 8.9GB of "bytecode".

There really isn't a layout algorithm but it does seem like overtime they began to contain it as to make it not platform dependent (thank god).

I think that the major achievement of webkit wasn't in the code they took from KHTML ;) but in the unit tests that were created and the robust amounts of work that went in to focus on making things pixel perfect. Prior to webkit a margin meant whatever the hell the damn implementers felt like. If it worked at all.

I should say much of the code in WebKit is no-mans-land, or essentially greatest common denominator work to make webkit portable to multiple architectures. That's why the linking takes so long and you can go from >1/2 GB of source code, 4GB of linker memory down to a 20MB object file.

>1/2GB of source code?

WebKit's entire Source directory is 247MB, and more than 100MB of that is ChangeLog files.

There's Servo from the Mozilla team, written in Rust -- not a pure functional language but an interesting step. (Not that there are strong guarantees it'll be less complex overall, though it may have other properties like memory safety)

This actually surprises me some. With how widespread the use of webkit is, I would have thought it was among the more cleanly designed pieces of software out there.

Anyone have any good pointers on an analysis of the code base? Any easy way to just "dive in" to the code other than the obvious checkout and look around?

> With how widespread the use of webkit is, I would have thought it was among the more cleanly designed pieces of software out there.

Sadly being widespread is no guarantee of clean design ;)

WebKit, like other browser engines, was originally designed in the 90's, and evolved over a long period of time. Again like the others, it's a large and complex C++ codebase, with all the good and bad that comes with that.

WebKit does have some advantages over other browser engines though, specifically it is the most embeddable, and I would say that is the main reason for its being widespread today.

This further hits my curiosity on whether or not there are any examples of "cleanly designed" systems that have "won" in the long run. Am I just growing cynical?

I still harbour feelings that something in the discipline of webkit offers lessons. I just don't know if I can see it myself. Outside of the lessons of brute force being a good tool. (Where I'm assuming there is a lot of force behind webkit.) (I do understand that they have a lot of tests that are fairly complex. Mayhap that is the true lesson. Get a good handle on how to run complicated test cases.)

I would point out Qt as a very well and cleanly designed project that "won" as in it's one of the if not the top GUI frameworks out there.

http://qt-project.org/

Is it actually all that widespread if you consider the full set of all apps on all platforms? The impression I get is that the majority of apps don't use any cross-platform GUI framework and instead use the native API of the platform - Win32 for pre-XP Windows, .NET for Vista+ Windows, Cocoa for Macintosh, GTK or KDE for Linux, HTML/CSS/JS for the Web, native Android for Android, native iOS for iOS, etc.

KDE is the community, not the toolkit. Most KDE specific widgets have been merged/replaced in Qt over time. KF5 mostly is a bunch of small libraries using Qt.

http://dot.kde.org/sites/dot.kde.org/files/kf5_big_0.png

Qt is also used by a few "big" software and as the default toolkit for BB10, Jolla and Mer (totaling a nice 1% market share ;) ). It is also seen as the successor to Motif for large scientific applications.

I think the causation runs the other way. When something becomes popular, there's pressure to improve it (or, arguably, "improve" it). That leads to more features, which leads to more code, which leads to a messier, more complicated codebase. This is all good for users - it takes complexity out of the usage of a feature and puts it inside the software, where it belongs.

The problem occurs because this process continues until the equilibrium point is reached, where it is no longer economical to add features to the software. Given the high usage and low marginal cost of software, this is usually when the software has become so inscrutable that nobody knows how to modify it anymore, at which point it is replaced by something else.

I wasn't trying to imply any causality. Merely somewhat surprised that it isn't better designed. We get into a fair number of arguments over "clean design" at work. I have grown weary of all of the "cleanly designed partial solutions" I have seen. Just as I've grown weary of complaints on how crappy a system is and how it should all be done from scratch.

It isn't even that I disagree. Just that "everything is terrible" is not a recipe for a happy day for me. For the most part, everything is bloody awesome. The details just aren't as clean as some would have hoped.

"...any examples of "cleanly designed" systems that have "won" in the long run"

The Linux kernel comes to mind.

Does this count as "cleanly" designed in the way most software engineering types put it forward? I mean, it flat out went against what was seen as "cleanly designed" when it was created. And even now espouses a monolithic code base over the microkernel designs.

That is to say, in my mind it was one of the counter examples to the idealistic "cleanly designed" solutions.

Just another reason I found it strange that Opera dumped their browser engine for webkit. I suppose it might look even worse..

There's two next steps:

A good build system/toolchain with GYP that generates Xcode, VS C++, and ninja project files. That would be mega awesome. QtWebkit already has some gyp files, probably could repurpose those.

Also, you can read up on embind on emscripten's website, you can use c++filterjs and the derived object WebCore.exp on the build to create javascript API's from the C++ export symbol table.

I just need to actually have a few days free to finish that and hopefully can have SOME functional demos from the API (that and hopefully I can throw away 90% of the symbols being exposed in the current version and cut the code size down :/)

These concerns resonate a bit with Google's explanation for Blink. Really excited to see what comes out of that.

I'm a bit late to the party here, but figured I'd chime in with my own experiences. I spent a many days staring into WebKit and can honestly say that finding your way around does get easier. When you are searching for things, it helps to have a good base grasp of what IS, and what is NOT WebKit. Webkit is many, many things, but people make some common mistakes about what it is. Unfortunately, to get anything done with WebKit, it requires a huge amount of work, and platform specific implementations and support, which add the the general confusion.

Reply on news.ycombinator.com