Hacker News Reader

Comment by thamer | Hacker News Reader

An engineering blog from the BBC published an update a few days ago on their migration to a serverless architecture, titled "BBC Online – A year with serverless": https://medium.com/bbc-design-engineering/bbc-online-a-year-...

They report 3.3B function invocations for 2.3B requests served, a total of 61,000 hours of execution time and 1,500 concurrent executions at peak.

Using calculator.aws, simply entering the number of requests at 67ms each (61k hours/3.3B invocations) with 128 MB of memory each computes a cost of only $1,107.

Then of course there's bandwidth, databases, file storage, and a lot more… but 3.3B lambda executions for $1.1k is really cheap. Not to mention that a client like the BBC would benefit from negotiated pricing with significant discounts. Even without discounts, serverless is often pretty cheap.

View on HN

The CPU throttling of the lower memory settings is often easily visible in a user-facing request, even simple requests just doing some DB IO. I rarely use less than 1024mb for anything

edit: the BBC review is horrifying:

> The page takes around 500ms to render and be delivered to the audience. In that timeframe we invoke around 30 functions. Around 150ms is spent running React to render the content to HTML

> we aim to personalise almost every page in some way — making it relevant for every user on every request

Good luck making perf numbers with all those cold cached personalized pages

I want to call out - performance numbers don't actually mean nearly as much as people have made them out to be.

This is a mostly read only web page. Half a second to load? You're barely going to lose anyone, if you lose anyone at all. Hacker News routinely runs me ~300ms to load and has zero personalization, Facebook takes over a second and a half before anything displays, as does Youtube (on a refresh, no less, so things should reside in cache locally!). Hitting a random person's LinkedIn page (once I've passed the verification, which is a whole different issues) takes 1.2 seconds. Etc.

Now, admittedly those are including the latency on my end, but the point is, no one is so meth addled that a page loading after half a second (or even a full second!) is going to have much effect on engagement. Even the studies that have been done (that I have some major issues with) only really start measuring anything significant well after a second or two.

The BBC article is only talking about generation time, specifically not download time (including linked assets), and we have no idea about dwell time.

It only takes a 3G connection a few miles outside a city to add another 500ms to that opening request. Say we're up to a second before some readable text appears, now we'd like to know how long the user will actually spend reading the text or waiting for images to load before navigating again.

Intuitively, I think that load time/dwell time ratio probably captures what those studies talk about better than just raw numbers. 1 second between 10 second TikTok video loads would be extremely noticeable, but barely worth mention if the user instead was spending 10 minutes reading e.g. a feature length news article.

My personal BBC reading habit regularly involves clicking into an article just to catch the opening paragraph and seeing which opening image they used (they rarely use the same for the thumbnail). The average is probably not as low as 10 seconds, but it's certainly something much less than 2 minutes.

Dwell time probably isn't a great way to capture it either. My pattern is quite "flicky" but I bet there is a spectrum all the way from "reads every last word" to "literally just loves to click". I guess latency becomes increasingly important for folk further along that spectrum

Oh, I know; measuring is imperfect.

My point is just that there's a lot of room before it even hits the numbers quoted by the various 'studies', and given those studies all tend to be from perspectives of advertisers and click through rates (i.e., giving people time to wait makes them go "wait, do I even care about this?", whereas this is someone explicitly looking to a news source), it hardly is the problem the parent presents

That’s only averaging 72 request per second. Yes they, scale to 1,500 requests per second sometimes, which is an advantage of lambda, but we are talking about pretty small traffic.

Still interesting though and glad it works for them.

And that's where serverless excels, highly variable loads.

You can certainly run a web server capable of 1,500 concurrent connections for under $100/mo.

Run it on Linode or something, they'll throw in 10TB of bandwidth which probably covers it.

Most of the serving is done by the CDN..

Right, but once you're in the territory of "everything for $100" or "everything for $1000", they're both the same so you want the thing with the organizational efficiencies of independent deploys and permissioning, resilience and scaling, high availability etc.

Two $60 Linodes in different data centers are highly availablish considering each one could easily do x10 those numbers. The thing in the link sits behind a CDN.

Certain kinds of organizational efficiencies frequently lead to "I'm bored lets use the new hotness that would look good on my resume for the next job", hence lambda.

Actually you don't need anything at all, this is a job completely for the CDN which has enough features they aren't using to do the same thing for no additional cost.

Hmm, I don't think that's the primary motivation. But the NomadList guy runs everything on one VPS and he makes $2m/yr so more power to you guys. I wouldn't do it because I'd want to be able to scale the team and keep their velocity high and I think clearer process-impact boundaries help.

For what's it worth, I've worked in Ad Tech at 2 m qps and now in HFT so I'm not averse to efficiency, but org efficiency is (in my experience) very important if you're scaling the company to the $100+ m revenue range.

I think if you're scaling the company to the $100+ m revenue range you'll be able to find a team that can handle either VPS, Lambdas, both or whatever else is required.

It really depends on what it is that you're actually, you know, doing.

I think that's a bit facile. What you're actually able to do is principally a function of your agility and I think a business's choice of tech impacts that. But far be it from me to prescribe. I think these things make us more agile and you think your things make you more capable. May the market favour the approach it does.

AWS has this way of making things more approachable. It is a subtle trap. Before lambda running a VPS was not considered rocket science. It may or may not come with scaling issues, depending on your actual application and the architecture.

Lambda lets you get going quicker but comes with its own idiosyncrasies, don't get lulled into a false sense of security. You're going to end up with a devops guy either way and they'll be able to handle either scenario.

Which one you should pick really really depends. But, all things being equal, it is better to shoot yourself in the foot than having AWS shoot you in the foot.

You gave your own counter-example. NomadList has a simple app, Lambda is the wrong choice for him. BBC link tells me their thing is so trivial it is moot - whatever floats their boat.

If I was doing some complicated EMR in BigCorp with many departments and stakeholders, slinging millions of files around S3 I'd pick lambda.

If I am setting up ad servers I would not pick Lambda because it is latency sensitive. 'etc.

Get yourself a devops guy and a proper architect and game it all out.

Think of your devops guy not as a server administrator but an advance AI that accelerates the velocity of your team to new peaks of productivity. They are not only debugging their code in production at 3am but your org processes.

You seem to have missed the aim of the game on their part is personalization. While it might be possible that they could do some client side processing, split the page up sufficiently, to then make multiple requests and assemble a "personalized" page, it's hard to say for certain that it would be. Certainly, it makes it less "personalized" and more "regionalized", which though being the case they describe, is likely a decision they didn't want to be locked into. For it to be truly personalized it likely needs to pull in some backend data.

Given that...you can't rely on the CDN being an optimization. You could still make the personalized calls through the CDN, if you wanted and had a clear caching policy, but something like "on load, assemble a page with real time updates on the stories this person expressed interest in" isn't going to benefit from a CDN.

Heh. OK for fun:

They have a very finite amount of new articles a day and a little box on the page with personalized headlines. All these articles, in fact all the articles on BBC of all time ever, comfortably fit into ram if you are so inclined.

Those little linodes have fast enough SSDs, try playing around and use one of countless stress loaders to get a feel.

I've served hundreds of thousands of requests a second from a single server using Varnish. A well tuned beefy VPS would do tens of thousands.

How to "personalize"?

For example Varnish has this thing: https://en.wikipedia.org/wiki/Edge_Side_Includes https://varnish-cache.org/docs/6.0/reference/vcl.html

It will happily read whatever cookie you set and stitch together a remixed personalized page for all the BBC users in the world without a performance penalty much faster than your lambda cold starts.

CDNs sometimes let you hook into Varnish directly or expose something similar to essentially do exactly this.

The aim of the game of the guy who wrote that blog post is to reinvent something from the 90s, but worse, and blog about it.

It's not just new articles; it's also updates to existing. The idea then being you could put together a feed of updates of articles you expressed interest in, plus recommendations and such; this isn't what they're doing now, but it may be their long term goal and this is their path to it.

I will 100% agree that what they describe in the link can be done trivially without lambda (they describe a single use case of regionalization, which doesn't even take anything special on the part of the CDN), but the point is I can easily see a future use case they're building towards, that I don't think those standards you mention will support.

I don't get why not. How do you think this was achieved before lambda? Or even before aws?

It is a news site not Facebook. The feed would only ever be a subset of all the articles in cache. It can itself be cached, there are only so many permutations. Different people will have different feeds, but there will be people with the exact same feed as you.

Crunch those recommendations however you want wherever you want and spew them unto S3 at your leisure. Have your varnish cache sit on top of that. Set your personalized subscriptions via a cookie. Varnish on each request will decide which one out of n (probably n<10000) feeds to include in your html response, in under 2ms.

This is all stateless. Have two servers in two regions. Done.

Maybe you'll reboot them every couple of years.

Been there, done that. Can confirm it works swimmingly.

It is much more fun though: don't be clever. Just cache-bust on change.

Reply on news.ycombinator.com