Hacker News Reader

Comment by seibelj | Hacker News Reader

Amazon has made scalable, performant, high-availability systems constructable by a 10 person team that serves millions or even billions of people. Before AWS it took thousands of people and billions in capital investment to do so.

Sure, google and Microsoft and IBM joined the party, but AWS was first and remains the best holistically. This is their moment of domination, and eventually something will knock them down, but they have made so many companies so nimble and powerful in ways that were impossible before. Go Amazon.

View on HN

>Before AWS it took thousands of people and billions in capital investment to do so.

WhatsApp Stats (2014):

- 450 million active users, and reached that number faster than any other company in history.

- 50 billion messages every day across seven platforms (inbound + outbound)

- 32 engineers, one developer supports 14 million active users

- $60 million investment from Sequoia Capital

Which they managed their own FreeBSD servers hosted on SoftLayer.

[1] http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...

YouTube (2008):

"YouTube grew incredibly fast, to over 100 million video views per day, with only a handful of people responsible for scaling the site"

- 2 sysadmins, 2 scalability software architects

- 2 feature developers, 2 network engineers, 1 DBA

"They went to a colocation arrangement. Now they can customize everything and negotiate their own contracts."

"Sequoia invested a total of $11.5 million in two separate rounds and was the only venture firm to invest in the company." [3]

[2] http://highscalability.com/youtube-architecture

[3] https://www.nytimes.com/2006/10/09/business/09cnd-deal.html

I'm pretty sure GP is mistaking valuation for capital. Serving half a billion or more people probably nets you a billion dollar valuation or more these days, but it in no way requires a billion dollars to provide that service in the vast majority of cases.

There is a sweet spot where cloud is good and provides some benefit but, once you're serving hundreds of millions of people and have double-digit millions in investment, you can probably do significantly better cost-wise rolling your own servers. Worst case, you just throw your own hypervisor management system on them and have most of the same features you got from a cloud service. If you're smart, you can probably architect it so you have on-demand overflow capacity from a cloud provider in case there's a spike you can't account for, which is the best of both worlds.

This is how we do it. Two on prem datacenters, one colo, and a handful of on-the-ready cloud providers. We serve far fewer users, but we also are getting 20 to 50k per user per year. Needless to say, at the scale we have cloud is out of the question except in catastrophic scenarios.

Back in the day, reddit was definitely serving a couple million users with like three staff and on-prem servers.

and yeah, they were down all the time, but that didn't seem to matter to their growth.

I've built high-availability systems that served millions with a <10 person team years before AWS even existed, at a time where our server racks had less combined capacity than my laptop does now, and the dual fridge sized storage array we used had less storage (and IO capacity) than the M.2 drive in my laptop does now.

The part of that solution which was related to making the system scalable was written by two of us, who also did other things (it involved a partionable backend storage service, and a user registration service, that combined to let us migrate users between servers to even out load and partition storage; everything else was stateless).

This idea that AWS is necessary to build to scale with small staff just does not match reality. My years of consulting also showed me that I'd earn more from clients who insisted on AWS - they typically spent far more time and resources on devops (and spent far more on hosting overall).

AWS is convenient, and it's great when you can afford it, but it's expensive and still requires substantial devops effort.

Go Amazon.

Indeed. Hopefully, soon they'll stop selling physical items in that online store they have and focus on their strengths, so that other companies, who might be able to do better at selling things that aren't so frequently counterfeit that I no longer buy anything from Amazon, can have a go.

>Before AWS it took thousands of people and billions in capital investment to do so.

Could you expand on this?

I can't tell if you mean to launch a service/company or if you're talking about some large scale.. thing.. I haven't heard of.

Instagram comes to mind. 13 employees and $1B acquisition.

Hard to imagine that without AWS.

WhatsApp? Similar story, no AWS.

Well, WhatsApp was started by industry experts in scaling. If you're making a point about functional programming, I'd tend to agree but from a business perspective I'd look to why Netflix still uses AWS: https://www.quora.com/Why-does-Netflix-still-use-AWS

Still they place FreeBSD running on prem hardware in the ISP PoPs as a caching layer because it absolutely makes sense. https://papers.freebsd.org/2019/FOSDEM/looney-Netflix_and_Fr...

I think people tend to forget how often WhatsApp experienced outages in the early days.

This actually speaks volumes. Yes, people forget. Yes, it’s possible to suffer severe growing pains and still get acquired for $$$$. No, you don’t need to start with everything-AWS to ensure 99.9-whatever% uptime. People forget.

An $1B acquisition may have nothing to do with any colossal infrastructure. Was it indeed colossal?

I don't know what you'd consider colossal but the migration doesn't sound fun: https://www.wired.com/2014/06/facebook-instagram/

Why? Sharding and caching blob storage and activity feeds for Instagram type sites is among the easiest category of sites to scale.

Not it didn't. This is hyperbole.

Moore's Law made it possible, not AWS.

Reply on news.ycombinator.com