How to Scale a System from 0 to 10M+ Users
blog.algomaster.io
blog.algomaster.io
1) His user numbers are off by an order of magnitude at least, as other comments have mentioned. Even a VM/VPS should handle more, and a modern bare-metal server will do way more than the quoted numbers.
2) Autoscaling is a solution to the self-inflicted problem of insanely-high cloud prices, which cloud providers love because implementing it requires more reliance on proprietary vendor-specific APIs. The actual solution is a handful of modern bare-metal servers at strategic locations which allow you to cover your worst-case expected load while being cheaper than the lowest expected load on a cloud. Upside: lower prices & complexity. Downside: say goodbye to your AWS ReInvent invite.
3) Microservices. Apparently redeploying stateless appservers is a problem (despite the autoscaling part doing exactly this in response to load spikes which he's fine with), and his solution is to introduce 100x the management overhead and points of failure? The argument about scaling separate features differently doesn't make sense either - unless your code is literally so big it can't all fit in one server, there is no problem having every server be able to serve all types of requests, and as a bonus you no longer have to predict the expected load on a per-feature basis. A monolith's individual features can still talk to separate databases just fine.
I've seen engineering orgs of 10-50 launch headlong into microservices to poor results. No exaggeration to say many places ended up with more repos & services than developers to manage them.
So whenever an inevitable common utility improvement is made, the effort of pushing out 100 repo releases for services no one has touch since Jim left 3 years ago is terrifying.
When there is a breaking change is going to be made and you HAVE to do the 100 releases, it's terrifying. Everyone says it never happens, but work on a project/team for 5 years and it does eventually, even once is enough to swear me off this "architecture".
To me it seems like microservices (or cloud, or whatever) is often overused for career/buzzword reasons. The engineers pushing for it aren't asking for your advice, they want to build an engineering playground - denying them the opportunity is unlikely to suddenly make them productive at driving the business forward with a simple stack when their original idea was to play with shiny tech instead.
The only way I see out of this is to have management buy-in to get the microservices and their developers out the door, replaced by more competent people.
This is a behavior I would say is very hard to manage out of people and should be screened for aggressively in interviews.
The most effective way to increase income for a developer is to join a place, rack up as many buzzwords as possible and leave after 2-3 years, using those buzzwords to secure a higher-paying role somewhere else. Rinse and repeat until you get a management position where you can use politics to increase your income instead.
If you want guys that use boring tech to drive the business forward you have to pay them upfront the money they’d otherwise make playing the above game. It still makes sense (an engineering playground is anything but cheap) but good luck getting an employer to pay anything above “market rate”.
I don't know if you can in general - by happy coincidence the engineer who had made that plan found his way to the door shortly thereafter
He started complaining to management that 50 CI/CD setups was his limit he could support.
He was absolutely amazed when I showed him he could combine endpoints into a larger logical service. 50 services became three, and it’s still three a few years later now.
While microservices force physical separation, they don't stop "Spaghetti Architecture." Instead of messy code, you end up with "Distributed Spaghetti," where the dependencies are hidden in network calls and shared databases.
Microservices require more discipline in areas like:
Observability: Tracking a single request across 10 services. Consistency: Dealing with distributed transactions and eventual consistency. DevOps: Managing N deployment pipelines instead of one.
For most teams Modular monolith is often the better "first step." It enforces strict boundaries within a single deployment unit using language-level visibility (like private packages or modules). It gives you the "Separation of Concerns" without the "Distributed Spaghetti" network tax.
According to my understanding this is one of the reasons why microservices were invented, to prevent shared databases?
I'm not sure if this is a discipline issue in the way that domain driven design, say, is a discipline issue. If you instrument requests with a global ID and point at tool at it then you're basically done from the individual team perspective.
Sure you can say e.g "this property wasnt set in this request while being processed by this service managed by this team", but why it wasn't set will inevitably need multiple teams, each doing in-depth analysis how such as state could've been caused because they always inevitably become distributed monoliths - the former is being provided by the instrumentation, but the latter isn't (and even the former is not perfect, as not all frameworks/languages have equal support)
"grug wonder why big brain take hardest problem, factoring system correctly, and introduce network call too
seem very confusing to grug"
> note, this good engineering advice but bad career advice: "yes" is magic word for more shiney rock and put in charge of large tribe of developer
Microservices definitely contribute to having a "large tribe of developer" to manage.
I feel a bit of sadness for people who had never used a bare metal server and seen how insanely capable hardware is today.
And how do you predict with certainty your "highest expected load"? And if you're in a space like ecommerce, where you have 1 week out of the year with x10 or x50 the load, I doubt it would actually be cheaper than using autoscaling. Especially today, with the costs of memory and storage. Not to mention that whenever you hit your load maximum, you have a few months of lead time to get extra capacity.
And FYI, "proprietary vendor-specific APIs" sounds very scary, but if you think about it for a few seconds, those APIs end at configuring an autoscaling group which is mostly about your min/max, and scaling rules. Yeah, it's proprietary, but it's 3-4 parameters to configure based on what you need, and from then little if any adjustment is needed. And you can take the same logic and port it to any other cloud provider within ~10 mins at most.
If you know when that week is, where's the problem in spinning up extra capacity just for that period, a week in advance?
Retailers, whether online or not, know with a very high confidence what their expected load is going to be a week from now.
I agree that in general the database is always going to be your bottleneck and autoscaling appservers is generally bullshit.
But let’s assume your DB is fine and you do actually need to scale appservers beyond what your provisioned capacity can allow, you can trivially spin up VMs at AWS or your provider of choice, Wireguard to your “VPC” at your bare-metal provider (or just SSH tunnel the ports) and run your appserver binary/container/etc on it.
Hell in a pinch you can do the above with your dev laptop and serve traffic from it. It will work and your customers won’t even be able to tell.
I didn't say you have to buy them. I said you can spin up extra capacity.
Modern hardware is fast, if you cannot fit more than 100 users (not even 100 concurrent users) on a single $50/month server, you're doing something very very wrong.
Even repurposed 10 years old fairphone[1] can handle more than that.
[1]: https://far.computer
Thank you stranger.
PostgreSQL and Elasticsearch clusters can be operated at a fraction of the cost of comparable managed services offered by the major cloud providers.
The idea that this necessarily involves excessive maintenance effort is nonsense.
The skills needed to use hyperscalers properly are better invested in fundamental sysadmin know-how.
It isn’t an apples to apples comparison. But, you trade some additional operational overhead for a whole lot more hardware.
(of course, say goodbye to resume points and your cloud provider conference invite. Question is, what are you trying to do? Are you building a business, or a resume?)
Then there's the implementation language category. interpreted, JITed vs. AOT.
And of course the workload matters a lot. Simple CRUD application vs. compute-heavy or serving lots of media, ...
Together those factors can make like 6+ OOMs difference.
Depending on usage patterns you can go higher.
I put well over a few thousand users on a single $5/m DO droplet. The backend was a single Go binary updating a postgresql DB (on the same machine) as and when users finished their mandated training material, keeping track of which slide they were last on.
The actual training slides were served statically.
Python is slow; I mean, like really slow. Java and C# requires extra RAM, and not just a little - 1GB minimum for any webapp you intend to build.
Jenkins, written in Java IIRC, was not able to run on that $5/m droplet even serving just a couple of users. A Django app I built also had trouble with acceptable response times with just a a few users too.
My little Go binary, OTOH, peaked at 300MB RAM usage and had no problem even with 1000 users all logging in at 0800 and rapidly running through their training.
Author having this on his github makes me even more suspicious: https://github.com/ashishps1/learn-ai-engineering
All of it aligns with https://en.wikipedia.org/wiki/Wikipedia:Signs_of_AI_writing
Maybe the images were made by hand.
- random bold emphasis that would make disney and marvell comics blush
- overuse of "one paragraph then bullet points"
- a lot of the bullet lists has a small bold prefix then one line for no good reason
- every section has a "why it matters"
- and then each section with an useless comparison table that are direct screenshots of ChatGPT/Gemini
I would not mind if the author indeed used his alleged insights in the domain, but as other have noted, numbers are way off, and are what CSPs want you to believe to sell more instances in Kubernetes. This does not inspires "I proofread the LLM output".
It's a shame because the article has some good advices, but also a lot of misled ideas that would make Grug scratch their head. No, you don't need Redis to have stateless applications. Having a load-balancing tier is as useful for resiliency than it is for scaling. Autoscaling is a trap. If you can afford it, start with the app and DB separated. Let your application perform connection pooling itself from day one, your framework knows more than PgBouncer how connections can be safely reused.
Overall, at a high level, the article is good and is a good outline on the order in which to optimize (sharding is dead last), but the details don't meet expectations.
You are not. The entire post is LLM slop. I am as baffled as you are how something like this could be on the front page of HN.
The senseless bolding of random words in the text, numbers that are way off and don't make any sense are the clues that the post is LLM generated and the blogger didn't even bother to proofread it.
https://www.pangram.com/history/a4492704-2897-4cfb-b921-b281...
I ran a 10k user classic ASP service on a VPS from Fasthosts, with MySQL 5.6 and Redis, and it was awesome.
Twitter famously had a "fail whale" but it didn't stop the company from growing. If you have market demand (and I guess advertising) then you can get away with a sub-optimal product for a long time.
Agreed, but there's still an element of survivorship bias there. Plenty of companies failed as they couldn't keep up with their scaling requirements and pushed the "getting away with a sub-optimal product" for too long a time.
Friendster might fit though: https://highscalability.com/friendster-lost-lead-because-of-...
If it’s just “sign up any time you want and go”, yes, it can go that way.
If it’s “join that waiting list” or “book a call” (for KYC purposes or whatever), you have a buffer.
If user count is more or less constant (most internal websites, for example), it’s probably not an issue.
And so on.
It is frustrating that the awesome article about Nonograms currently has less than 70 upvotes. But this pure LLM slop has 100+ upvotes and counting! What would it take to stop this LLM slop infestation?