Hacker News Reader

Comment by dpweb | Hacker News Reader

If you have an engineering mind and care about such things - you care about complexity. Even if you don't - user experience matters to everyone.

Have you ever seen something completely insane and everyone around doesn't seem to recognize how awful it really is. That is the web of today. 60-80 requests? 1MB+ single pages?

Your functionality, I don't care if its Facebook - does not need that much. It is not necessary. When broadband came on the scene, everyone started to ignore it, just like GBs of memory made people forget about conservation.

The fact that there isn't a daily drumbeat about how bloated, how needlessly complex, how ridicuous most of the world's web appliactions of today really are - baffles me.

View on HN

Honestly, I think "a daily drumbeat about how bloated, how needlessly complex, how ridiculous most of the world's web applications really are" pretty much describes every HN conversation on any article with even a remote connection to web technologies.

Jeez, you aren't kidding. It's as though, if we bitch enough about the technical deficiencies, the economic and social circumstances that cause them will, I dunno, vanish into thin air I guess. This topic is pretty much just red meat here

Well, bitching isn't that great. But as a web developer myself, it has affected the way I work. Like, there was an article just the other day about how to achieve "lightning fast page loads". I shared it around the office and as a result, 4 people have added new steps to their workflow to help reduce page load latency on our client sites.

The author knew the article would get good reception here, because of how much people complain about page latency, and I knew that page latency is a thing that's important to more people than just me, for the same reason. (My clients mostly don't seem to care, for some reason, but I bet their customers do).

So yeah, having these public conversations can make a difference. We just have to stay positive and constructive.

This is a great point. As much as it can be a bit annoying to see the same conversations over and over again, and as much as we make fun of supposed groupthink, it's all a long slow process of consensus building and idea dissemination.

Thanks for reminding me of that; I was previously sitting here thinking, "oh geez, here we go again!".

Thankfully web developers are sharing the goodness and distributing 100MB+ executables using NodeJS & CEF so desktop applications can be insanely large too :^)

What would be super awesome would be a daily drumbeat about how to slim down and simplify applications, with working, open-sourced code.

Here, I'll beat a drum a little. Maybe it will inspire somebody.

I just wrote this tiny text-rendering engine, mostly yesterday at lunch. On one core of my laptop, it seems able to render 60 megabytes per second of text into pixels in a small proportional pixel font, with greedy-algorithm word wrap. That means it should be able to render all the comments on this Hacker News comment page in 500 microseconds. (I haven't yet written box-model layout for it yet, but I think that will take less time to run, just because there are so many fewer layout boxes than there are pixel slices of glyphs.) http://canonical.org/~kragen/sw/dev3/propfont.c

The executable, including the font, is a bit under 6 kilobytes, or 3 kilobytes gzipped.

Impressive!

Thank you, but I don't think it's impressive! It's still nearly an order of magnitude slower than memcpy(). But if we want simpler systems, I think doing experiments like this is a big part of how to get there.

The fact that there isn't a daily drumbeat about how bloated, how needlessly complex, how ridicuous most of the world's web appliactions of today really are - baffles me

Well there is, or at least there used to be. The reason native apps in mobile became so popular is exactly because web sites are bloated. It might not matter in the PC but in a device running on batteries it matters a lot.

The real problem of web development is JavaScript. It’s a relic of the past that hasn’t caught up with times and we end up with half-assed hacks that don’t address the real problem. We need something faster and way more elegant.

I absolutely disagree. JS is plenty fast for >99% of the things being done with it. The DOM, and manipulations of it, is the slow part.

JavaScript isn't the problem, but it seems to always take the blame. And its not even slow.

Here are a couple of real culprits:

* Advertising / analytics / social sharing companies. They deliver a boatload of code that does very little for the end-user.

* REST and HTTP 1 in combination. A page needs many different types of data. REST makes us send multiple requests for different kinds of data, often resulting with those 60-80 requests mentioned above. We could be sending a single request for some of them (e.g. GraphQL) or we could get HTTP 2 multiplexing and server push which completely fix that problem.

* JSON. Its simple but woefully inadequate. Has no built in mechanisms for encoding multiple references to the same object or for cyclic references. Want to download all the comments with the user info for every user? You have a choice between repeating the user info per comment, requesting the comments first then the users by id (two requests) or using a serialisation mechanism that supports object references (isn't pure JSON)

* The DOM. Its slow and full of old cruft that takes up memory and increases execution time.

> * Advertising / analytics companies. They deliver a boatload of code that does very little for the end-user.

That's not quite fair. They subsidize the content for the end-user. Perhaps that's a crappy status quo, but in many cases without the advertising and analytics the content wouldn't exist in the first place.

Perhaps. But maybe, if you add googleads, amazon adsystem, moatads, rubicon project, taboola, scorecardresearch, krdx etc [1], then you start wondering why are users using adblockers, so you add pagefair, and at that point you want to find out what works better so you add optimizely... maybe, just maybe, at that point, you're actually losing money because your page is so damn slow, rather than getting more because of your efforts to perfectly monetise it.

[1]: Just copied them from coldpie's news site screenshot comment. That was about 1/2 of them, there were many more.

I remember when I was young, Television was sponsored by Advertisers: One of my favourite shows, was made possible because of Advertisers. To bemoan Advertisers in my mind is to say that part of my childhood should never have existed, so I think greater, more focused criticism is required.

Do you ever wonder why the landscape became the way it did?

If a TV show were five minutes of content, and 55 minutes of advertising, I would stop watching it. And yet, that's the approximate breakdown for many websites between content and advertising. And they wonder why readers complain.

They know why readers complain. They wish you understood their perspective better.

A significant problem is that content costs a certain amount of money to be produced, and web content is unable to command those prices.

Ad fraud is a big part of it, and some of the companies in the best position to solve it (like Google) are benefitting so handsomely from ad fraud that I can't imagine them stopping it.

Ad blockers will hurt legitimate content producers, but because people defrauding advertisers don't use ad blockers, they'll continue to make money.

I'm sorry, but watching an ad isn't just a payment like you do with money. It's a rape of your mind. It needs your personal data, to rape your brain deeper. So i block ads, not to see content, but because ads don't deserve to exist.

> It's a rape of your mind.

This is perhaps the most ridiculous comment I have read this afternoon.

Read more about ads, their perception by our brain, how they work and how they are designed to bypass our consciousness. Next, understand the concept of metaphor. And then, come back... or not.

A rape of your mind?

Wow. Well, I'm glad you have your feet anchored down here in reality and aren't exaggerating at all.

There's actually a lot of momentum on this from with the IAB (standards body for online advertising). The goal is to separate content from analytics. The result would enable publishers to prioritize loading the content before loading analytics.

Hmm, cool, didn't know they were working on it. That should help.

If it degrades the user experience so much that the page is unusable, it's not serving the intended purpose. Users will block the ads or not bother waiting for the page to load. Some ads/buttons/analytics code is worth it, but most content sites don't seem to think about the tradeoffs at all.

I'm increasingly of the mind that:

1. Advertising is the problem.

It's creating technical problems. It's creating UI/UX problems. It's creating gobs of crap content. It's creating massive privacy intrusions and security risks. And for what? Buzzfeed?

2. Ultimately, the problem is the business model for compensating informational goods. Absent some alternative mechanism (broadband tax, federal income tax applied to creative works), I don't see this changing much.

https://www.reddit.com/r/dredmorbius/search?q=broadband+tax&...

3. Micropayments aren't the solution.

http://szabo.best.vwh.net/micropayments.html

Is it not an option to just stop visiting these sites?

Think of the children. I mean authors.

I'd like a system under which creators of quality content would be equitably and fairly compensated. The present system fails this.

What if they already are? I can think of a handful of newsletters that command a three-figure annual subscription price. The people who buy them must regard them as useful and of sufficient quality. It could be that the rabble of news sites drowning in ads are mostly schlock, and that their business model is one of attempting to monetize some of your least-attentive moments.

I'm increasingly of the mind that:

1. Advertising is the problem.

It's creating technical problems. It's creating UI/UX problems. It's creating gobs of crap content. It's creating massive privacy intrusions and security risks. And for what? Buzzfeed?

https://www.reddit.com/r/dredmorbius/search?q=broadband+tax&...

3. Micropayments aren't the solution.

http://szabo.best.vwh.net/micropayments.html

> Want to download all the comments with the user info for every user? You have a choice between repeating the user info per comment, requesting the comments first then the users by id (two requests) or using a serialisation mechanism that supports object references (isn't pure JSON)

Or you send it like

    {
      "users": [
        {
          "id": "de305d54-75b4-431b-adb2-eb6b9e546014",
          "name": "Max Mustermann",
          "image": "https://news.ycombinator.com/y18.gif"
        },
        ...
      ],
      "comments": [
        {
          "user": "de305d54-75b4-431b-adb2-eb6b9e546014",
          "time": "2015-10-15T18:51:04.782Z",
          "text": "Or you can do this"
        },
        ...
      ]
    }

Which is a standard way to do it that works here, too. Because you can have references in JSON, you just have to do them yourself.

Alternatively, I would think that gzip does a good job of factoring out repeatedly embedded user objects.

yes, obviously – gzip even does that explicitly, replacing repeated text with references.

But it still takes more power for your server to go through gzip every time. And it will take more RAM on the client to store those objects.

Yes I agree that gzip will probably fix this single-handedly. The main cost here is very likely network transfer, so gzip will do great.

Don't forget it takes time to parse large json blobs.

And to serialize it from whatever the in-memory representation is.

Something similar has been standardized as well with the JSON API specification (although it adds its own weight to the message as well, it does address this problem): http://jsonapi.org/format/#document-compound-documents

Its a great solution! I'd use dictionaries for faster lookups though.

But not exactly pure JSON. Client-side, you need to attach methods (or getters) that fetch the user object to the comment. I suppose you could just attach get(reference) that takes this[reference + '_id'] and looks it up inside the `result[reference]`. m:n relations will be harder though.

Otherwise you can't e.g. simply pass each comment to e.g.a react "Comment" component that renders it. You would also have to pass the user list (dictionary) to it.

Well, you could process the JSON on client side.

    response.comments.forEach(comment =>  comment.author = response.user[comment.author]);

Preferably, even, you could do it during rendering so it can be garbage collected afterwards.

Well, that counts as further deserialisation in my book. At least if you set up a convention to automate it for all kinds of objects. Otherwise you'd have to do it manually for every type of request

> * The DOM. Its slow and full of old cruft that takes up memory and increases execution time.

I'm not going to say that the DOM is wonderful but … have you actually measured this to be a significant problem? Almost every time I see claims that the DOM is slow a benchmark shows that the actual problem is a framework which was marketed as having so much magic performance pixie dust that the developer never actually profiled it.

Good point. Lets see:

http://jsfiddle.net/55Le4ws0/

http://jsfiddle.net/nem6tnv1/

Initialising about 300K invisible DOM nodes containing 3 elements, each with 3-character strings is ~15 times slower than initialising an array of 300K sub-arrays containing 3 elements, each being a 3-character string.

Additionally, paging through the created nodes 2K at a time by updating their style is just as slow as recreating them (the console.time results say something different, but the repaint times are pretty much the same and this is noticeable on slower computers or mobile.)

Thats a single benchmark that raises a few questions at best. But I think React put the final nail in the coffin: their VDOM is implemented entirely in JavaScript, and yet its still often faster to throw away, recreate, diff entire VDOM trees and apply a minimal set of changes, rather than do those modifications directly on the DOM nodes...

It's not useful to compare the DOM to a simple array which doesn't do most of the real work which you need to do. Comparing rendering that text using canvas or WebGL would be closer if it also had a layout engine, dynamic sizing, full Unicode support, etc.

Similarly, React is significantly slower – usually integer multiples – than using the DOM. The only times where it's faster are cases where the non-React code is inefficiently updating a huge structure using something like innerHTML (which is much slower than using the DOM) and React's diff algorithm is instead updating elements directly. If you're using just the DOM (get*, appending/updating text nodes instead of using innerHTML, etc.) there's no way for React to be faster because it has to do that work as well and DOM + scripting overhead is always going to be slower than DOM (Amdahl's law).

The reason to use React is because in many cases the overhead is too low to matter and it helps you write that code much faster, avoid things like write/read layout thrashing, and do so in a way which is easier to maintain.

Most of the elements are hidden via display:none - there is no layout work to be done for them whatsoever. The rest of the overhead seems to lie entirely in allocating a lot of extra data per node.

Also, total time to repaint seems to be equally fast whether we're recreating the entire 2K rows from scratch or changing the display property of 4K rows. That seems concerning to me.

But yes, a more fair comparsion would be to write a WebGL or Canvas based layout engine in JS. If its less than 4 times slower (its JS after all), then the DOM is bloated.

Also, remember that my position is that most applications are limited by other factors before you hit the DOM.

I certainly would expect that you could beat the DOM by specializing – e.g. a high-speed table renderer could make aggressive optimizations about how cells are rendered, like table-layout:fixed but more so – but the more general-purpose it becomes the more likely you'd hit similar challenges with flexibility under-cutting performance.

The most interesting direction to me are attempts to radically rethink the DOM implementation – the performance characteristics of something like https://github.com/glennw/webrender should be different in very interesting ways.

> Well there is, or at least there used to be. The reason native apps in mobile became so popular is exactly because web sites are bloated. It might not matter in the PC but in a device running on batteries it matters a lot.

I think you'd find a much easier case made that its because:

1) discoverability and distribution through app stores,

2) access to native features that took years to be enabled in the mobile web (for example access to photos..)

3) ability to make games (access to graphics, etc -- this also kind of hampers the case being made about "bloated" sites when people seem very willing to download 100MB games)

4) access to ease-of-payment (CC info stored on people's phones that you can in app payment, vs having to type info into a web form).

So we could try to focus on clear issues the web has vs native, or I guess we could design yet another language that is theoretically faster thanks to >insert pet feature here< and hope that fixes the web.

> 3) ability to make games (access to graphics, etc -- this also kind of hampers the case being made about "bloated" sites when people seem very willing to download 100MB games)

There's a huge difference between downloading a 100 MB app once (plus occasional background updates) and waiting to load a 100 MB web game every time it falls out of cache.

Plus the web loading delay happens constantly while you're trying to use it. You wait on an app download exactly one time: when you buy/install it.

Games also often have loading screens. I think the real counter there is that there is a big difference between waiting to play a 3d game vs waiting to read a news article.

There's also a big difference between loading something out of flash storage and loading it off the internet.

My 4G connection right now is speedtesting at 46.68 Mbps download.

An iPhone 6 benchmark I found on MacRumors can read at 6720 Mbps (840 MB/s) [1]

Given the choice, I'd take the option that's 140x faster.

[1] http://forums.macrumors.com/attachments/img_0005-png.514599/

We AABN tested injecting delay into rendering the MSN home page and found a direct correlation between amount of delay and the abandon rate. People are not tolerant of delay. Here is the team I worked on: http://www.exp-platform.com/Pages/default.aspx

"people"? What people? This OP's experience seems to indicate that there is a lot of patience in "Southeast Asia, South America, Africa, and even remote regions of Siberia".

It sounded more like patience was relative to expected wait time. If you expect something to take 20 minutes, then a 2 minute wait isn't much at all. If you expect something to take 2 seconds, then a 10 second wait is quite irritating.

I'm certain you're right but we need to test that :)

the way I read OP was the abandon rate was ~100% previously in areas with high latency/low bandwidth, and his experiment had a non-zero abandon rate that was shrinking as word spread. Perhaps I need to read it more carefully.

I took this screenshot yesterday. This was a page showing a news article (text). http://i.imgur.com/hmFaW3M.png

Most of those are on my /etc/hosts blocklist. Or would be were I aware of them. Worthless crap.

BTW: /etc/hosts + dnsmasq, for Linux, is amazing. (dnsmasq reads /etc/hosts and will block entire domains if listed as same).

For Firefox users, is this better than NoScript for any reason?

I think it could make pages load marginally faster, because NoScript only stops scripts from executing, where a hosts blocklist would stop the scripts from ever being downloaded.

I think, not totally sure how NoScript works.

Nah, NoScript blocks them at load-time, too.

I can't say whether it's better or worse. It's different. With NoScript you still hit "remote" servers when it comes to other resources, like images.

For Firefox, the real comparisons are Request Policy and ublock (origin) in advanced mode (for chrome, umatrix). These do full third-party host whitelisting per domain. So every time you visit a website on a new domain, by default all requests to third parties are blocked. Then you spend a few minutes working out which ones are required.

By comparison, noscript simply blocks javascript from third parties. It does include a number of anti-xss heuristics though.

uBlock and uMatrix are two separate extensions. Both are available for both Chrome and Firefox.

Putting hosts (and domains) under control of /etc/hosts and dnsmasq means that there's little likelihood of traffic reaching them from your browser (though Web hosts could provide back-end data transfers).

It's also possible to directly address hosts by IP, though unlikely (Web protocols such as virtualhosts would fail).

I'm strongly favouring uMatrix for now. It takes some tuning, but you have fine-grained control over CSS, images, scripts, XHR, frames, and other bits, by domain or host.

Aggregators and CDNs confound things a bit (Akamai, Amazon's cloudy thing.)

Oh, why are you blocking them? They are trying to enhance your experience on that website. /s

This is the problem exactly - it doesn't have anything to do with the language ... too many damn bells and whistles and too many damn analytics / ad libraries. If you're delivering content, you should focus on delivering the content. My computer shouldn't nearly freeze trying to read a recipe, or the weather, or a simple news article, etc ...

The fact that there isn't a daily drumbeat about how bloated, how needlessly complex, how ridicuous most of the world's web appliactions of today really are - baffles me.

I feel Jakob Nielsen used to majorly play this role amongst Web developers back in the day, but for some reason I haven't seen him come across my radar for many years now. The Web performance experts now are better than ever but I don't get the sense of the majority of the industry hanging off their every word as seemed to happen with Jakob 10+ years ago.

I get weekly newsletters from him. Still good stuff.

Unfortunately, computing has always been that way. Remember the mantra "Intel giveth, Microsoft taketh away?". Developers quickly get spoiled with faster hardware, more bandwidth, more RAM, etc.

I'm not saying it's right. In fact, I would prefer if everyone took a look at their apps much like the author of the article. I'm just saying, this isn't a new phenomenon.

> baffles me

IMO pretty clear why, winner-takes-most economics emphasizes dev speed over quality, and then we need to maintain compat with it

There are reasons for it: it takes more work to write an efficient, simpler application with the same functionality. Developer time is expensive.

Well damn.

I'm in Bangalore and being forced to use mobile 3G for now. The data charges are atrocious and each page load saps an MB.

Maybe you have taken these steps already, but uBlock Origin works great, and NoScript works okay, in Firefox for Android. They may get your page size down by significant fractions.

Reply on news.ycombinator.com