Hacker News Reader

> stop using CDNs for jQuery/js libs, those https connections aren't free - they're much more expensive than just serving the same script from your existing connection

Do you have a source for this? My understanding is that, in real usage, it is cheaper to load common libraries from a CDN because in a public CDN (for something like jQuery), the library is likely to already be cached from another website and has a chance to even already have an SSL connection to the CDN.

Obviously 60 separate CDNs is excessive, but I don't know if the practice altogether is a bad idea.

this has been true in my own testing (with plain SSL/TLS, not even SPDY/HTTP2/QUIC), but you can read another analysis that reaches the same conclusion here: https://thethemefoundry.com/blog/why-we-dont-use-a-cdn-spdy-...

another important reason i personally don't use CDNs is the privacy of my users.

>another important reason i personally don't use CDNs is the privacy of my users.

How high up the privacy concern is using a CDN vs having cloudflare in front of you or everything being in AWS or using a browser by an ad company?

> having cloudflare in front of you or everything being in AWS

sure, sometimes it's unavoidable when there is real value added in using these platforms. at the end of the day most of us have to entrust our infrastructure to some hosting provider. I trust a provider I pay (with a known privacy policy) more than a public CDN (with a privacy policy that allows them to make money in unspecified ways)

but yes, centralized ssl termination for distributed systems is an issue that's difficult to mitigate WRT privacy. maybe via https://en.wikipedia.org/wiki/Multipath_TCP ? i don't know enough here.

> using a browser by an ad company

this is up to the user

jQuery was a poor example in the original comment. You almost 100% have jQuery cached if you're getting it from CDN. But this only works for extremely widely used libraries (jQuery, Bootstrap, etc.). It drives me nuts when I see tiny little libraries in Github bragging about being on a CDN. They just don't have the usage for it to be efficient.

Which of the following versions are cached?

3.1.1, 3.1.0, 3.0.0, 2.2.4, 2.2.3, 2.2.2, 2.2.1, 2.2.0, 2.1.4, 2.1.3, 2.1.1, 2.1.0, 2.0.3, 2.0.2, 2.0.1, 2.0.0, 1.12.4, 1.12.3, 1.12.2, 1.12.1, 1.12.0, 1.11.3, 1.11.2, 1.11.1, 1.11.0, 1.10.2, 1.10.1, 1.10.0, 1.9.1, 1.9.0, 1.8.3, 1.8.2, 1.8.1, 1.8.0, 1.7.2, 1.7.1, 1.7.0, 1.6.4, 1.6.3, 1.6.2, 1.6.1, 1.6.0, 1.5.2, 1.5.1, 1.5.0, 1.4.4, 1.4.3, 1.4.2, 1.4.1, 1.4.0, 1.3.2, 1.3.1, 1.3.0, 1.2.6, 1.2.3

from which of the following providers:

- https://developers.google.com/speed/libraries/

- https://code.jquery.com/

- https://cdnjs.com/

- https://www.asp.net/ajax/cdn

- ...

And over which protocol (http/https cached separately)....

Using a CDN for common libraries may help, but doesn't always, and is something you should measure rather than just assume. The situation where a CDN actually hurts performance is one I've seen periodically at different clients.

When people talk about serving jQuery, or J. Random JavaScript library, from a CDN it means the specific version of jQuery (or whatever) that they're using. There's literally no guarantee that the specific version you need will be in any given user's browser cache, and this is exacerbated if you loading multiple libraries from a CDN, or from different CDNs. If your CDNs serve files with low latency then it may not be a big problem, but not all CDNs do. Slow responding CDNs will slow your page loads down, not the reverse.

Moreover, if you're serving over HTTP2/SPDY there's even less likely to be a benefit to using a CDN. Again, it's something you need to measure.

One area where a CDN (e.g., Cloudflare) can benefit you is by serving all your static content to offer users a low-latency experience regardless of where they are in the world, but that's rather a different matter from serving half a dozen libraries from half a dozen different CDNs.

> ... something you should measure rather than just assume

How do people do this?

My obstacle with this point is each CDN would have a different impact in each locale due to the locations of their points of presence, and CDN-ing each resource would a different impact based on the sites that particuar individual had visited.

Measuring it in any useful way in advance of a change would be really hard unless I'm missing a trick, and measuring it in hindsight would require the kind of performance logging that PaaS-level apps don't have access to.

my recommendation is to first serve everything yourself over TLS (minified, bundled, gzipped). if you can, run HTTP2. then measure.

if this does not deliver the perf you expect (chances are that it will), then look into CDNs.

if you need to serve media that needs a lot of bandwidth (expensive) then look into CDNs.

if you need some form of real-time comm like WebSockets or WebRTC that actually requires low latency, look into distributed systems (amazon, google, azure, cloudflare?, etc..)

Sure, my question was how people measure.

your best bet is https://developer.mozilla.org/en-US/docs/Web/API/Performance

combined with either IP2location or Geolocation apis.

This would be extremely useful data to have and when I last looked I found little to go on. For example, how great would it be to have an extra column on a page like this with cache hit rates? https://cdnjs.com/libraries

Nice to have these free hosted assets, but I suspect that a lot of these are an additional download and not actually cached by the majority of visitors. Just having a rough estimate would be helpful when choosing whether to directly serve an asset or use a CDN.

You can use JavaScript timers to measure how long things take to load and infer how many of the resources come from cache, etc.

I'm a webdev. I had to speed up web e-mail client, written in angular.js. Besides splitting huge request into couple smaller, async ones (huuge speed improvement) I've also switched downloading files from CDN to app server itself - that gives additional 4sec (sic!) shorter load time.

Can you explain how you achieved the 4 second difference using the webserver over the CDN? That seems orders of magnitude faster than I would have expected. I thought you would have been looking at 100ms one way or the other.

At my current client I'm pretty sure that they've got a Raspberry Pi 1 as a proxy server. So all request to outside their network are horribly slow.

Also, the cache size on mobile devices especially on iOS is ridiculously low. I don't see my cache storage exceed more than 10 MB even though I use my browser a lot. High chance that you are _not_ going to find your common CDN libraries there.

In theory public CDNs are the solution. In practice it only works if a critical mass of websites use them.

As others have pointed out, the version matters. There are a gazillion different versions of jQuery out there and many websites are very slow to update them if ever. So you only benefit by visiting multiple sites that use exactly the same version.

Additionally there is a ton of different public CDNs. Now, jQuery itself has solved this by providing an official jQuery CDN and pushing it to developers. In fact, the first result for "jQuery CDN" is their own website.

I'd say jQuery is probably the least bad example of this. If you visit enough websites there's a non-zero chance you'll have a CDN cache hit for jQuery on a few of them.

The problem is that jQuery rarely comes alone. Even jQuery UI is less widely deployed than jQuery and thus less likely to be cached from a CDN. But once you get into plugin territory or third-party libraries all bets are off.

I'm fairly certain that jQuery is the only library that has some realistic chance of benefiting from a public CDN. Everything else is probably asking for trouble (if only because you're adding unnecessary single points of failure).

> 1.22MB embedded youtube players

A guy who I used to post with wrote a new forum for us all to post on (woo splinter groups). It's pretty cool. One of the things it does is serve a static image of the underlying youtube and then load it on click. When a 'tube might be quoted 7 times on a page - that's a pretty useful trick.

I'd just assumed this was a standard forum feature and then I opened a "Music Megathread" on an ipboard and holy shit loading 30 youtube players was painful.

yes, you can lazy-load the youtube player, there's also a jquery plugin for it [1]. but this setup requires 2 clicks to play on mobile (because google restricts the player from autoplaying onload in mobile devices due to bandwidth/data-usage concerns). the first click will appear to do nothing as it loads the 1.22MB player, so delivers a crap experience.

you're much better off just serving an html5 <video> tag. you can likely fit your entire video in mobile quality in that 1.22MB :D

[1] https://github.com/tylerpearson/lazyYT

It's an issue with a poor UI design, not lazy-loading though.

I'd still prefer a video link, and leave it to my client to negotiate. I have never enjoyed viewing embedded videos in web pages (content aside).

Actually I don't even read the articles anymore when on mobile. I just use HN, and hope somebody posts a TL;DR, or some relevant comment that gives some more information about the article. Only if this is not the case will I consider clicking on the article link. It's pretty sad actually.

I secretly wish there was some way that allows us (as a community) to collaboratively "pirate" articles, perhaps as a torrent (IPFS perhaps), so we only have to download the ascii text.

Try turning off javascript in your main mobile browser (e.g. safari) and turning it on in your secondary one (e.g. chrome).

I use Brave as my primary mobile browser because it allows me to have JavaScript disabled but trivially enable it when I need it. The experience is massively better – on LTE the average page load is under a second except maybe for a video or lots of images, and you avoid all of the user-hostile pop-ups, app nags, etc.

(I subscribe to most of the sites which I read regularly so I don't feel guilty about blocking ads but I really hope someone successfully makes a Google Contributor-style network where a trusted browser serves only static image/text ads, as much resistance as that will trigger)

I usually only see lynx and friends mentioned as jokes, but 'elinks --dump <url> | less' really can be a huge readability improvement for some articles.

Back in the day we'd try and get page sizes down to less than 100k.

The Internet isn't fast for everyone. I (in the UK) have no 3G signal, let alone 4G and my broadband speed is pitiful - but it will do. There is nothing I can do to ramp the pipe speed up. I do end up turning off JS and images a lot of the time, because otherwise it ills me.

As a web dev, I don't care for bloat. So I find it particularly irksome, and currently it's enough to deter me from going mobile. Once I'd have dreamed about having a modern smartphone in my pocket with any Internet connection, but the friction currently today puts me off. The UK was recently slammed for its retrograde networks.

An IMHO interesting unit of measure is the DOOM(s), JFYI:

https://twitter.com/xbs/status/626781529054834688

As in "my site homepage is less than 1 Doom" or "that crappy site is more than 3 Dooms" ...

This is only slightly related, but I've noticed HN comment pages take a second or two to load when there are many comments (500+). Page sizes are not unreasonable (100-200 kb). What is the cause for these pages loading so slowly?

Example 900+ comment page: https://news.ycombinator.com/item?id=11116274

Example 2200+ comment page: https://news.ycombinator.com/item?id=12907201

The server is taking a long time to render the response. In the first link, 650ms or so. Open the network tab in your browser developer tools to see the details.

Slashdot's solution to this was to render the HTML on write, instead of on read. Whenever someone posted a new commend, a static HTML file would get generated, so that it could be rapidly served to readers. Movable Type works (can work?) much in the same way, though for blogs.

I know that caching achieves a similar result and is an accepted component of a modern architecture, but I wonder if we wouldn't be better served by implementing these sort of "upfront caches" in more software.

  Whenever someone posted a new commend, a static HTML file would get generated

It's essentially still a cache, with the same concerns/issues - mainly cache invalidation - as a HTTP cache.

An on-disk cache of that sort has other issues though. For example, what if a request is received whilst the file is being written? Does the system deliver a half-written HTML page? You can introduce locking or atomic file operations (write to a temporary directory, then mv the file into the correct location: mv is an atomic operation), but this adds more complexity to the cache logic.

One of the benefits of a HTTP cache is that they generally have a default graceful failure mode: if the cached data doesn't exist, request it from the backend application server.

The issue doesn't seem any different - assuming you have concurrent requests, you also have to make sure the cache doesn't return a result until the whole document is written to it.

That's a fair point - I didn't consider that issue, because using a standard HTTP cache, that's taken care of.

It's just become somebody else's problem (the developers of the HTTP cache), but it's fair to say it's still an issue to solve.

> literally cannot consume the internet without uMatrix & uBlock Origin

hear hear. and on mobile, its painful because I can't have those (windows phone at least). planning on buying a DD-WRT compatible router soon so I can do some kind of router level ad-blocking and let me browse on the phone again

PS: opera mobile for android has a built in adblocker

yep, i use opera mobile and also am rooted so use a hostfile blocker (AdAway).

uBlock Origin exists for Firefox Mobile btw, but i prefer opera's speed/ui.

A Raspberry Pi running Pi-Hole is what you want. https://pi-hole.net/

Ok, that's even better. I can get a pi for less than a new router, build my own case. If only my router had a usb port, I could even power it off that

I use uBlock in Firefox for Android. There's also AdAway to block domains in the /etc/hosts file.

Or just configure your router to use an ad blocking DNS (host your own or use one of those public DNS servers). Works with any router as long as it allows you to edit the default DNS config. This helps a lot when browsing on home wifi connection. Too bad you can't edit DNS config on mobile device without rooting/jailbreaking.

It's more general than just webpages, but applies to everything under the name “Jevon's Paradox”(https://en.wikipedia.org/wiki/Jevons%27s_paradox).

Basically: more power -> more resources can be analysed in the same time, and not faster to answer.

I am with you, but I don't believe that this solution will be ever used by the majority of developers.

Browser caches should be bigger. They also should be more intelligent. It does not make sense to evict a library from cache if it is the most popular library used. Maybe having two buckets, one for popular libraries and another for the rest.

I think that it would help if script tag had a hash attribute. Then cache could become more efficient. But without the first part it would be useless. Example:

  <script src="https://cdn.example.com/jquery.js"
  sha256=18741e66ea59c430e9a8474dbaf52aa7372ef7ea2cf77580b37b2cfe0dcb3fd7>
  </script>

Or different syntax (whatever I'm not W3C):

  <script src="https://cdn.example.com/jquery.js">
  <hash>
    <sha256>
      18741e66ea59c430e9a8474dbaf52aa7372ef7ea2cf77580b37b2cfe0dcb3fd7
    </sha256>
  </hash>
  </script>

I would like to make an experiment, but as I am not experienced with webdev as much, it could take too much time for me. Test all major browsers with fresh install and default settings. Go to reddit or other links aggregator and load in sequence several links in same order on every browser. Check how efficiently cache was used. I would expect that after 10th site is loaded nothing would remain from the 1st one. Even though the same version of some library and maybe even link to CDN was used.

I am amazed how quickly fully static pages work even after I am on capped speed mobile connection (after I use 1.5 GB packet).

EDIT: The most helpful thing would be to have good dead-code removing compilers for JavaScript.

Check out the list of security problems you'll need to solve to implement hash-addressable caching:

Subresource Integrity Addressable Caching https://hillbrad.github.io/sri-addressable-caching/sri-addre...

Yeah the problem really isn't the technology, it's our poor use of it.

Some files could be hashed and cached locally, such as jQuery files and fonts. I don't know why we have to load it every time from the web, it's the same library for a large number of sites. Just add a hash tag in the <script> to make sure it's the same file.

When JQuery is served by the official CDN, it's given a cache-header giving it a lifespan of 10 years.

the bandwidth isnt the problem.

your browser still has to make the https connection to the cdn and request the file with a Last-Modified or Etag header so the server can return a 304 Not Modified response. This is the real cost, not the download size itself.

I'm not sure how often browsers choose to do this, but if you refresh a page, they all will.

In general, browsers shouldn't even revalidate the file as long as it's younger than the max-age lifespan.

They may choose to "evict" it from the cache, but that's generally based on a policy of recent usage and needing to free up/prioritise disk space for more recent assets.

I'm now interested to see how different browsers handle refresh/force-refresh - whether they revalidate a file that's still considered fresh, and whether they set request-headers to bypass upstream caches too. Maybe an experiment for the xmas holidays :-)

last time i tested this was a couple years ago.

i could not find a reliable way to make the 304 requests disappear using any available cache-related headers :/

There is also Cache-Control: immutable [1] which prevents browsers requesting the given file even when the site is refreshed. It's experimental though.

[1]: http://bitsup.blogspot.de/2016/05/cache-control-immutable.ht...

Currently only supported by Firefox (as of 49.0), according to Mozilla [1].

Interestingly, it's being used by the BBC [2], who have reported a 10.8% reduction in 304 responses as a result [3]. Google have also stated their intention to use it (comment 11 in the same Chromium ticket).

[1] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ca...

[2] https://mybbc.files.bbci.co.uk/notification-ui/2.6.8-mvt//sv...

[3] https://bugs.chromium.org/p/chromium/issues/detail?id=611416

Another alternative would be service workers, which are (partially) supported by Chrome and Firefox according to [1]

[1]: http://caniuse.com/#feat=serviceworkers

The use of CDNs is not primarily for speeding up page load times, but rather to offload bandwidth from the web server. Low budget Web sites don't always have money for server farms and are severely limited by how much bandwidth they can serve. One post to HN can take them down. Free CDNs are the poor man's approach to this problem.

Right. CNN now has over 50 trackers and other junk blocked by Ghostery.

One gripe: the canonical CDNs may have the library already cached from your visits to other sites, which is faster.

I wish that web browsers would use content addressing to load stuff and do SRIs. If I already loaded a javascript file from another url, why load it again?

That is too big of a breaking change to do on the current web. There is a whole new initiative for that though - https://ipfs.io/

This, this, a thousand times this.

There's a site I read, really like and financially support, but which has some pretty terrible slowness & UI issues. It's so bad that they've started a campaign recently to fix those issues. But when I check Privacy Badger, NoScript and uBlock, there's a reason that it's so terrible slow: they're loading huge amounts of JavaScript and what can only be called cruft.

Honestly, I think that they'd come out ahead of the game if they'd just serve static pages and have a fundraising drive semi-annually.

The browsers should have stayed HTML/CSS but then someone had this idea to try to compete with native applications....

but they can compete when written properly, at least for business/productivity apps.

though not for anything requiring direct hardware access, AAA games included.

I sort of agree, i just think JS and content from other domains should have required user prompts.

and https by default should have never allowed off-domain or insecure/mixed content.

We can dream but, even if it had been that way, I suspect it would have taken just one bad browser (probably IE) to ruin it for everyone. They'd advertise it as "no annoying prompts", everyone would flock to it and other browsers would have to follow.

>but then someone had this idea to try to compete with native applications

You say as if that's a bad thing. Just wondering, for instance, do you use some standalone maps software instead of Google/Bing/Openstreet/... maps?

Ever heard of GPS devices, mobile phones or tablets?

I always use native software on my own computers, unless I don't have another option and have to go the web app instead.

Same applies to projects I work on, if I am allowed to choose.

No, I have not heard of GPS devices, mobile phones or tablets. Thank you for introducing me!

Testify Testify Brother Testify!