“You don't need this overengineered goo for your project.”

EDIT: To expand on it - they had to manually build a lot of what could be much easier handled today, and thus had no incentive to change later. Interestingly enough, SO has a lot of moving parts distributed over multiple servers, even if all of those servers used to fit into 1-3 racks per DC.

Either way is quite baffling how common this kind of comment is, almost a decade reading Hacker News and it pops up constantly.

I mean these politics are pushed by rich Americans and foreign interests, surely they have the means to start their own hosting platform. It's the same that advocate in favor of businesses rejecting customers because of free market.

At a previous employer, we had a k8s cluster with a bunch of machines that were designed to a) load a filesystem kernel module inside the container (yes inside, not outside), b) mount /dev from the host in the container with Docker, and c) mount hard drives from the host /dev inside the container using the "mount" command.

In a twist that should surprise no one, those containers don't work well. And they failed to work in crazy, confusing ways for which there is no documentation to troubleshoot, because who in their right mind would do something like that?

I've had better luck in places that have a Platform as a Service team that owns the k8s infra. They generally have a lot more pushback to say "no, you're not going to do that on our cluster" which helps to tamp down some of the crazier ideas.

* Maybe a bit more/less, because it's not clear to me if DB RAM is per server, or per cluster. Likely server, as on other servers. There is also no data on how big is their haproxy.

If someone says SO runs on a couple servers this might be about the number actually necessary to run it with full traffic, not the number of servers they use in production. This is a more useful comparison if the question is only about performance, but not that useful if you're comparing operating the entire thing.

Also 23 is very much a couple for a company and application of that size. It's not uncommon to see several hundred or thousands of nodes deployed by similar sites.

That is still doable with mid-90s era hand management of servers (all named after characters in lord of the rings).

Not that you should, but you could.

And the growth rate must be very low and pretty easy to plan out your O/S upgrade and hardware upgrade tempo.

And it was actually possible to manage tens of thousands of servers before containers. The only thing you really need is what they now call a "cattle not pets" mentality.

What you lose is the flexibility of shoving around software programmatically to other bits of hardware to scale/failover and you'll need to overprovision some, but even if half of SOs infrastructure is "wasted" that isn't a lot of money.

And if they're running that hardware lean in racks in a datacenter that they lease and they're not writing large checks to VMware/EMC/NetApp for anything, then they'd probably spend 10x the money microservicing everything and shoving it all into someone's kubernetes cloud.

In most places though this will fail due to resume-driven design and you'll wind up with a lot of sprawl because managers don't say no to overengineering. So at SO there must be at least one person in management with a cheap vision of how to engineer software and hardware. Once they leave or that culture changes the footprint will eventually start to explode.

But it does seem to have most of the features of something like Reddit.

The usual story. Everything works, until it doesn't.

If you are a huge corp with good engineering you can have people dedicated to understanding k8s and then it kinda makes sense. They can spend time to learn it really well so that they have necessary chops to deal with problems when they happen.

On the other hand, if you are smaller company, you are more likely embracing this new idea of developers running everything including k8s, you are in for a trouble.

They will know how to make it work but that's about it.

Because if you need to learn everything you actually learn nothing very well. And there certainly isn't enough time in the world to learn everything in development.

My philosophy is applications must be built for when it breaks and it is unacceptable to run an application with a team that will not be able to fix it if it breaks.

**

Couple of years ago I joined a small group of teams of developers (together about 40 devs) who together maintained a collection of 140 services, all with same or very similar stack (Java, Spring, REST, RabbitMQ).

They had trouble delivering anything to prod because of complex dependencies, requirements, complex networking, complex process to find out where stuff broke between 7 layers of services between the original user call and the actual execution of their action.

I rolled my sleeves and put everything in a single repo, single build, single set of dependencies, single version, single deployment process, single layer of static app servers.

I left after the team was reduced from 40 to 5. There was no problem delivering anything (we had 60 successful deployments in a row when I left) and the guys who were left admitted they are bored and underutilized because of how easy it is to make changes to the system.

These were still the same guys that were advocating for microservices. From what I heard they are not advocating for microservices anymore.

Can microservices be done well? Sure they can. But it takes additional effort and experience to do it well. Why make your life difficult when it is not needed?