Hacker News Reader

Comment by karmajunkie | Hacker News Reader

The only thing in cloud that's worse than a surprise bill is a surprise outage.

Depends. For my hobby project that hasn't been monetized, being charged $100k is way worse than it going down.

If it were critical infrastructure, or monetized in a way that brought in revenue to cover the charge, then maybe I don't want it to shut down despite skyrocketing costs, but that's hardly the only situation you could be in.

But more importantly: just let the customer decide! Let them decide whether there's a threshold where an outage is less costly than the hosting, and what that threshold is

Nobody is going to charge you $100k.

Perhaps that is true, but the stress and anxiety from seeing 100K bill is real and has an impact.

https://www.forbes.com/sites/sergeiklebnikov/2020/06/17/20-y...

Just in case you don't know, the person you're replying to is the author of the blog post, and they mention in it that being stressed out by (potential) unexpected large bills is something they are aware of.

Although even then, "we will only threaten to charge you $100k without meaning it" isn't much of a reassurance.

If you're stressed over lighting $100k on fire, then you're not the target market. /s

(ignores the multiple hobbyists who accidentally got charged $100k)

Getting charged is not the same as paying those charges. I'm willing to bet real money that all who, in good faith, went and asked not to pay those bills ended up not paying them (if they didn't ask, that's a separate issue).

My belief, and you can correct me on this if you have evidence to the contrary, is that nobody actually pays those bills.

Someone paid those bills to somebody. AWS has costs incurred from it (support, opportunity, etc) and they paid them.

From that point of view, the cap seems more like a common-sense self defense feature that the cloud provider would want to implement. But we have cloud providers in this thread saying they don’t want to implement caps, so, I dunno.

From my technical understanding from a few friends who may or may not know what they are talking about, the caps issue has to do with the processing delay between billing events being emitted and understood. That by the time the billing events have been processed and action taken, the damage would already be done in all but the most extreme cases.

Secondly, the best solution is to simply stop everything. But now the customer has to cold-start their entire infrastructure, which may actually cost more than paying the bill.

Thirdly, it is likely that customers will set a billing limit and then forget about it years later. Suddenly, they've got a complicated infrastructure setup spanning the globe. They finally hit a scale where they hit their billing limit that they had completely forgotten that Bill configured in their early days (who doesn't even work there anymore). Suddenly, the entire global infrastructure is shut down in the middle of the night.

That's the gist of what my friends said.

I'm just going to refer people to this comment.

This made me think. There is usually some "hard" cost limit X that you cannot / don't want to afford, so terminating the service is preferable.

There is also usually some "soft" limit Y < X that you don't want to exceed, and don't plan to exceed, but you'd rather pay >Y than face an outage.

But a hard limit would have to be set to X to avoid that outage, and if it gets exceeded, you'll face a bill of X and an outage.

So what a customer would actually need is to specify both X and Y, with the rule: If the cost would exceed X, then terminate it early so the cost doesn't actually exceed Y.

Sounds complicated to implement, but then, the current practice of waiving the bill is complicated too if you tried to formalize it.

(For the sake of this discussion, I'm ignoring all the technical difficulties of terminating a high-availability service at all.)

It would, of course, be just mean and unscrupulous for a cloud vendor to look at the number you have set as being ‘the absolute most I am willing to pay for this service’ and then optimize their pricing offer to you specifically to make sure they go right up to that line and no further.

I didn't mean to imply that. A hard limit at X that stays inactive <X, and at >X, leaves you with a bill of X and an outage is the easiest approach from a technical side: Terminate the service when X is reached, and bill exactly what was provided. It is something you would instantly come up with when asked to implement a cost limit, and you don't for a second put yourself in the customer's position.

Of course cloud vendors do put themselves in the customer's position, and that's why they say that customers would not be happy with a limit, even though they are asking for it.

Is this a soft limit or a trajectory prediction? I think there isn’t such a thing as a soft limit. Nobody wants to spend any money really right? But you need to spend some to avoid losing service. That’s just a cost you don’t like but need to pay.

I definitely get the idea of: I don’t want to spend X so if it looks like I will, terminate service at Y. But I think that’s a special case of the general situation, I want to know how much I’m on track to spend, right?

But I don’t know much about this at all. My whole experience was accidentally getting my own personal self a $500 AWS charge and then deciding they cloud services were dumb.

> Is this a soft limit or a trajectory prediction?

I don't know. I just tried to frame the problem from a customer's point of view, because cloud vendors' statement that customers would not like a limit is (IMHO) limited by their POV. Customers do want a limit, but not the way that cloud vendors would implement it. I think a huge part of the problem is understanding what exactly it is that you need when you use a cloud service. (This is varying from customer to customer, and from service to service, of course. You usually have important services that must be running, and others where an outage would be unpleasant but not critical.)

> Nobody wants to spend any money really right? But you need to spend some to avoid losing service. That’s just a cost you don’t like but need to pay.

That is not the issue. From a customer's POV, I would be ready to spend extra to keep the service running, but there is a limit where I'd prefer an outage because I can't bear that much. There are two problems with that: First, the limit is blurry. Second, a simple hard limit would leave me with a huge bill AND an outage. I would want to be able to choose one of those evils, not be left with both. And these two problems compound.

> But I don’t know much about this at all. My whole experience was accidentally getting my own personal self a $500 AWS charge and then deciding they cloud services were dumb.

I don't think they are dumber than the alternative. If you run your own hardware, you have a hard limit in both cost and computing power. You could technically get that with the cloud too, but it is not usually offered because it doesn't really solve the problem, but neither does it for for your own hardware.

That said, it would be nice if the major clouds would offer a "hard limit" option, but it really only works for "unimportant" applications that are cost-sensitive and can take an outage.

Reply on news.ycombinator.com