How Twitter deploys its widgets JavaScript | Hacker News Reader

How Twitter deploys its widgets JavaScript

blog.twitter.com

54 pointsdoppp23 comments

"Many bugs manifest themselves during deploys, especially when there are a large number of code changes."

I'm intrigued by this hole in the process. At Twitter scale, how is it that these bugs make it past testing, QA, CI, and staging?

I know the Dijkstra quote[1], but still: Twitter AFAIK has as organized and talented a code practice as any company of that size needs in order to do any programming and maintenance, and still they get the kinds of problems that even freelance clients gripe about costing them thousand$/hour. I also realize that I'm playing a little No True Scotsman here, but again: still.

1. http://www.brainyquote.com/quotes/quotes/e/edsgerdijk201165....

Do they clean up and minify first?

The devs at Pinterest and TripAdvisor are jerks about this. Try following this one:

    <script src="https://www.jscache.com/wejs?wtype=certificateOfExcellence&amp;uniq=847&amp;locationId=307144&amp;lang=en_US&amp;year=2016&amp;display_version=2"></script>

After the redirects, they serve you a file that is unminified and with 20% comments. I'm not sure if the engineers at these places hate their users, or are just incompetent.

EDIT:

They do: http://platform.twitter.com/widgets.js

While not minifying is a bit unprofessional, I'm not sure wasting 1.4 KB counts as "hating your users" in this world of large retina images.

> "hating your users"

You're right, I went to far. Not hatred, disregard and carelessness. Contempt maybe.

It would take them a couple of hours at most (testing included) to add minification to their build process if they have one (a this stage, one can't be too sure since they don't follow basic web dev best-practice).

> 1.4 KB

That is for one badge. This is even more madness; they make you download the same js file from a different URL for each badge you want to display. Resorts like to show as many badges as they have to show that they've been highly ranked for a long time. A client of mine who wants to show her little resort has been the topped rank in her area for the past five years has to add 7KB of JS to do it.

> This world of large retina images

Not outside the first world, and tripadvisor is a global company supporting multinationals and tiny little 2-hut resorts alike. It's very likely they made the same assumptions that you did... which is basically my point... that they have no respect for their users (resorts and web devs). A little internet cafe in Timbuktu doesn't have 7KB of data to waste.

I have recently come across a bug in relation to this particular javascript widget. Once they dropped the requirement for widget IDs here:

https://blog.twitter.com/2016/embedding-twitter-timelines-ju...

Widgets no longer worked. So basically if you had widget IDs on data elements in HTML, the code no longer executed. Forced upgrade of the frontend code to match the JS requirements.

How does this work along with browser caches ?

Cache-control headers. Browsers won't cache indefinitely unless you ask them to.

Am I insane or are cache-control headers unreliable? I always set cache-control: no-cache on my S3 files but still get complaints about people getting old versions with broken pages as result.

Some CDNs don't always listen to those (I've noticed cloudflare especially has a problem with them if I don't add the files to the ignore list in their control panel), maybe that's it?

I don't rely on them. We set query parameters on new JS code to definitively bust caches.

src="foo.js?v=1", then src="foo.js?v=2", etc.

I'm sure people consider this a hack, so I'm open to hearing other suggestions.

It's not a hack at all for resources you control - in fact, it's encouraged to do so in order to use very long lived Expires headers while still being able to deploy new versions of your app.

It doesn't work for unversioned library code though where you want to automatically upgrade all users of your code as is the case with Twitter it looks like.

Problem with these is that many caching proxies consider all resources with a query string to be dynamically generated and won't cache them at all, which in the case of js/css/etc is bad

Sure, but that's easily solvable by versioning assets in the filename itself. That's how the `filerev` task works for both Grunt and Gulp.

I use this as well, works like a charm.

you are either insane, using them wrong, or following some bloggers guide instead of reading actual specs.

Had the same thought. Looking at the headers they serve the file with a mere 30m expire time.

I was wondering if the experimental group gets a cache time of 0 so they can immediately be rolled back.

they probably set no-cache, which gives them the worst performance. and don't even use it for their benefit since they decide what to serve based on dns, which is always cached in the os and the browser.

found that write up extremely comical.

You can no longer embed their widgets on platforms that allow iframes but not script tags, like most learning management systems.

This has now been posted 5 times: https://news.ycombinator.com/from?site=blog.twitter.com

But not discussed yet, so the reposts are ok. Please see https://news.ycombinator.com/newsfaq.html.

(We do say a 'small number' of reposts, which 5 is probably over, but not that much over.)

Probably because no one cares.

That probably underestimates the randomness of what gets noticed here.

Reply on news.ycombinator.com