Fun trivia: for the longest time, lichess.org - which by any measure is a high traffic, hugely interactive web app - was run on... a single server with 8 CPUs and 24Gb RAM[0]
[0] https://lichess.org/forum/lichess-feedback/hardware-of-liche...
[0] https://lichess.org/forum/lichess-feedback/hardware-of-liche...
Our latest MVP - https://www.code-scope.com lets one interactively analyze and document GBs of source such as linux kernel.
It runs on Digital Ocean's $5 instance, build just with Jquery - bootstrap4.
The app binary that runs code-scope.com website is < 10MB including all the frontend and backend code.
Good old tech such as jquery works very well for us.
Complex js front-end tooling is a choice, not a requirement for develop web app.
I still annoy people by talking about my time at the high frequency trading firm, because working there was a revelatory experience. They were operating at a scale and velocity far beyond any place I've worked at, with lower tolerance for downtime, and very little hardware, relatively speaking. The traders had fun, of course, but it was frankly kind of boring to work there as a techie, what with everything ticking along so smoothly all the time.
And the way they got there was through an absolutely fanatical devotion to the KISS principle.
What I do is put my database on a droplet and my files on another and it can handles millions of requests. Put them on the same machine and I need to increase ram higher than the cost of multiple small droplets.
If you ever reach those limits consider setting up a load balancer.
All the web queries are responded in <<< 1 second - so far....
There was a lot of homegrown software where others would use a library. That was deliberate. If you need five features, but choose an off-the-shelf package with 5,000 features, pretty soon you'll find that the list of features you're using has grown to 50, then 500, and so on, even if you don't actually need any of it. If a toy's in the box, the kids will play with it.
Maybe one thing to try is to focus on imposing constraints, and making people live within them. RAM, # of CPU cores, whatever. Software is like a gas. It will naturally expand to fill all available space.
I've come to think that scale out is a self-fulfilling prophecy. If you choose a solution that's designed to support scale out, then all the mechanisms and limitations (yes, limitations) it needs to adopt in order to make distributed processing work will impose a burden of overhead that you will need to offset by scaling out.
I haven't seen any certificates that replace sweat and hard work yet, except in organizations where they cannot keep track of individual developer performance and simply count certificates.