Comment by pjc50 | Hacker News Reader

Comment by pjc50 | Hacker News Reader

Parent Full thread

Autoscaling is a very particular solution to a very particular problem. Being a startup is like trying to make a hit record: everyone wants to become an overnight success. Nobody wants to be an overnight success and then have to turn customers away, or give out a bad impression because their site can't cope.

So a lot of effort goes into producing a site that could be "Facebook scale" if it became necessary. Although since this is difficult (and expensive!) to test, it's likely that some panicked manual intervention will be required if it does take off.

It's really worth asking "what is the cost of scaling" next to "what is the cost of NOT scaling, and what are the chances of it happening?". Is your organisation even able to benefit from rapid scaling - would it be profitable? Or are you going to end up like that luggage company screaming at its customer service reps to work dozens of hours a day?

If your cloud hosting cost goes from free-tier to $100,000/month in one month, and it's real traffic, is your business actually going to be able to pay that invoice?

Is the static overhead of scalability better for your use case than what a normal PC is capable of, which these days can be quite a lot: https://adamdrake.com/command-line-tools-can-be-235x-faster-...

(Been thinking about this since I mentioned the SRCF earlier, which was a single-server host for years: https://news.ycombinator.com/item?id=21741789 )

Cost is a very important factor... But you focused on the high cost end..

The cost of running a production server is something small startup businesses need to budget for... And you don't want to run your production website on a 5-10/month shared server. If you design for the cloud/serverless, you can get your costs down to near-$0 for light usage... But yes, using things like lambda mean a possible bill shock if your traffic grows faster than your income can recover it.. but if you have an advertisement on the page, it probably will cover the execution cost if it's lean enough.

In another example, I inherited a CICD pipeline that cost $300/month and could run 2 of our larger test suites simultaneously.

I converted this to AWS Spot instances and now support thousands in parallel and I pay under $170/month ( https://ldoughty.com/2018/01/gitlab-multi-runner-w/docker-ma... )... But to your point about "what if" costs of this, yes, there's a limit on place for this setup so it can't go over 20 or so... And it is important to consider the impact of auto scaling with regards to the bill... For a small business, AWS effectively doesn't have a way for you to really stop "Bill attacks" or legitimate high traffic on API Gateway/Lambda short of you taking down your site.

My recommendation is: ideally, don't expose something that costs you much too run unless you can recoup the cost from that action (paying customer, advertising).. your homepage should be static or cached in a CDN until you're ready for that large bill.

Dedicated servers are actually quite cheap. I pay $60/month to run one of my products that services thousands of paying customers across North America (High performance native TCP and UDP based game like servers) and the performance is quite predictable

Indeed the servers can be cheap! The inherited configuration used a redhat image with support that added a premium cost on top of being a slightly beefier server to run a few dozen containers simultaneously for testing. With the configuration change we optimized the choices and dropped support.

That said, it really wasn't much work to drop the 6pm-6am costs to effectively 0 unless a developer decided to work late (it automatically scales up)

Please research opportunity cost. The total time you spent to optimize the service was probably in tens of hours.

Multiply that by the amount of money you cost your employer per hour and you'll realize that your "optimization" actually resulted in a net loss.

I'm fully aware of opportunity costs, the investment was made because an investment needed to be made regardless to stand up 2-3 more servers. Doing so would have raised costs to perhaps $400/month if we simply copied existing setups (plus 2-3 days of work, probably, since we had not revisited CICD in over a year)

So by that, I've saved $5,000 in pure server costs. It probably cost me 10 extra man hours to do this way... But I also resolved a bug that was not terribly worth fixing before but took up 30 minutes of time every few months.. and improved the security by taking the servers off the public web.. however you want to quantify security improvements.

I always look at opportunity costs and the automation cost/benefit! The article was researched and written on my own time for my own enjoyment (so this was actually "free" from a business perspective, but my responses factored it in as if I did it as paid work. Just clarifying in case someone wants to claim I'm publishing "work"

For me a huge benefit of serverless is the ease of development speed.

Google's Firebase Functions are ideal for REST endpoints, they bill by the millisecond instead of AWS's 100ms (I think?) pricing.

If you buy into Google's ecosystem entirely they provide you a lot of functionality to get off the ground running. E.g. their (free!) auth service integrates with security around functions. In less time than it takes to look up the YAML for Nginx config you can have a proper authenticated set of REST endpoints up.

Now that said all I have my CPU intensive stuff running on a fixed price VM.

Exactly!! I still think that people do not think enough about the cost of NOT scaling nowadays. And especially, what is it going to cost when I can't scale and I have to change course. I recently heard a podcast where they said: "choosing a distributed databases is probably premature optimization right?", I was pretty surprised. A startup often sells a better user experience. Although less important for US startups since US is a big market, international start-ups won't get many clients further away from their country if the latencies increase signficantly so distributed databases is something you might desire anyway. Secondly they might indeed end up losing clients if they break down due to scaling issues. And at that point, if they want to keep their clients, the options are limited, 1. they'll have to go through a risky deploy to change the database, 2. try to scale it yourself by creating multiple replicas with all problems and technical depth that might bring, finally maybe still losing your clients if you fail and maybe ending up to pay more due to overprovisioning. 3. Accept that clients complain or leave. I would choose FaunaDB or Spanner for my startup personally.

Reply on news.ycombinator.com