Hacker News Reader

Comment by moksly | Hacker News Reader

The thing I really want out of these services is the ability to set a payment cap. It’s probably never going to be an issue, but I have anxiety, and I can’t sleep easily knowing that if I fuck up, if someone sinister abuses my application or whatever I may be stuck with a giant bill.

View on HN

That's not a bug... that's a 'feature'.

Interestingly, AWS will not cut you off for non-payment (we had an issue with finance and were 250k in the red by the time we got the first 'Is there any issues over there?' email)

And even as an employee, if you tell them over and over again that customers keep asking for that feature, they disagree with you over and over again.

This sounds like a short term 'thinking'. If there are a few stories on the internet (and there are some in this very thread) about developers getting burned by this 'feature', it would turn off a lot of potential users in the future. It would make sense to be transparent and give tools to strictly control usage/billing - both in terms of trust and money.

It is. I'm using free there to build and host my projects. If/when I exceed the free tier I'm not making a billing account unless I can set a max bill. I'll sooner rewrite everything to run on bare metal that has a fixed monthly cost.

I pay my $5 for linode (expanded swap to 2GB, it's SSD backed) and just run Caprover... I'd rather not deal with AWS's billing (even their billing dashboards are delayed by 24 hours)

I've been using gc/firebase. It's very easy to use, with acceptable documentation. I've been able to get dangerous quickly without even trying to. If I weren't on the free tier I would've racked up unaffordable bills. Instead I exhausted my API limits and had to wait. Perfectly acceptable behavior for development. I'll also add some logic to the finished project to handle resource exhaustion. Should I start exhausting resources,and it's not a bug, I'm not going to continue with infinitely scalable/infinitely billable gc. I guess if target customers are companies with 9-12 figure budgets getting a 9 figure bill wouldn't be the end of the World. If I got a high 4 figure bill I'd have to declare bankruptcy.

If you have a payment agreement, GCP won't cut you off provide your account is in good behavior in the past.

If your only payment method is credit card, IMHO allowing a past due account to continue generating cost is a very risky move and an easy target of abusing (imaging stolen credit card etc).

Disclaimer: I work for Google Cloud as SRE and have first hand experience dealing with past due account (it was highly manual and not automated).

You can deploy with a —max-instances flag to prevent infinite scaling. This has been a feature since the Cloud Run beta and has been in GAE forever.

One issue in GAE is that it can take 24 hours to update the billing cap. So if you happen to get a ton of fantastic press coverage one day, your site will go down and there is literally nothing you can do to fix it.

Seems unrelated to capping instance counts, which I assume would go into effect very quickly.

(I work on GCP but not this product)

Set budget alerts. And check what's going on if they start going off.

Or think again if you really need 'infinitely scalable bla blah' for personal project on a small budget.

Real answer here.

aws lightsale and digital ocean are great alternatives for personal projects

I do not want the failure mode to be the expensive case.

Here's a story about why you are right to be anxious: https://hackernoon.com/how-we-spent-30k-usd-in-firebase-in-l...

This was discussed on HN (237 comments): https://news.ycombinator.com/item?id=17661391

These stories are everywhere. A couple months ago, it hit an associate of mine pretty hard, he moved a small python monitoring and statistics application off a laptop in the office to AWS. A couple weeks later he came back and discovered that it had burned up a few thousand $'s in storage and transfer fees for what normally was a couple hundred MBs of data a day being passed around and a some database updates.

Since it wasn't really worth debugging what went "wrong" it got chalked up as a learning experience and moved back to the laptop where its happily running. If it dies/whatever its no big deal, just a git clone;./runme; and it rebuilds the whole database and creates another web facing interface.

The IaaS guys are masters of marketing. They have convinced everyone that their offerings are less expensive and more reliable, which repeatedly is proven to be false in all but the most dynamic of situations. In this cases its saving $7.99 a month over a unlimited site shared hosting godaddy/whatever plan. Just in case it might need to scale a lot, in which case your going to be paying hundreds if not thousands in surge pricing.

No thanks...

Not sure what happened to your associate but this sounds way, way out there. I run a fairly resource intensive SaaS on AWS (lots of background jobs generating data) and we barely go over $600 a month.

People should not be scared of by these anecdotes, however true they may be. It’s perfectly possible to run a very cost effective business on AWS.

That's great. But you are always just one unfortunate bug or mis-configuration away from having it happen.

Sure, to most people it will never happen. But the risk is always there. Reminds me of https://en.wikipedia.org/wiki/Survivorship_bias

It's possible to write bug-free code, but would you bet your financial future on your ability to write bug-free code?

Starting to think the reason we arnt working for google/aws is we make mistakes. Whereas engineers at those companies just dont make mistakes, therefor they assume that billing caps are not necessary. Such is our lot in life.

I think the bigger difference (other than a potential skill difference) is rather that they're more willing to take these risks. The ones that are good at it and succeed will rise to the top!

Bug free is a not realistic but if you are a programmer and definitely if you are a tech founder you definitely should be willing to bet your financial future on writing reasonably low bug code.

If you are a programmer, I kind of find the comment puzzling; maybe I am reading you wrong, but you seem to be saying that you are writing code for some company while being happy to commit it and not care if the company loses money if your code is bad? As you would not bet on your work yourself, but you do not mind your employer paying you for it and as such betting some of it’s financial future on your work? Sorry if I misunderstood your comment.

When you work for somebody else (or with somebody else) then you try to do as good of a job as possible, but the ultimate responsibility still lies elsewhere - might be your boss or could even be the group. There will be other people who interact with your code and might spot errors. There will be people who are trained, in some capacity, to figure out ways to mitigate against accidentally generating very large bills. It is exceedingly unlikely that these points hold true for a solo developer working on a main project, let alone a side project.

Even if you are hyper competent and can probably get all of this correctly, you can't rest easy. You simply don't know whether you did everything correctly or not. Just one dumb mistake can saddle you with an enormous bill.

This is just like gun safety: don't point your gun at anything you're not intending to shoot. Mistakes happen and the consequences of it can be catastrophic.

You are right, but at some level you are thinking 'you can do it' right? Otherwise you would be pretty miserable I would imagine. But I agree with the rest you wrote. You meant it slightly more nuanced than I read it!

Virtual cards with limits are an answer to this. Works very well.

Not really. You're still responsible for the bill even if your payment method is capped. They don't just forget about it.

If you never pay then expect some aggressive account shutdown, bans across all connected user accounts, and calls from debt collections.

You are absolutely right; no idea (thinking hard what service I was thinking off now) why I typed that. I was thinking of something entirely else. No-one try this; bad advise; they will indeed come after you. Not

It seems absolutely within amazon's technical ability to allow you to prepay for usage, and then evaluate your use on a per-hour basis.

I have a side project that uses AWS at the moment, and while stuff like serverless RDS instances are really cool, it scares me that somehow amazon is going to have a bug which empties my checking account. I've read as much of the documentation as I can find and have done everything I seem to be able to to prevent this from happening according to AWSs documentation, but it sill worries me.

In fact here is a banking feature I'd love to see: per merchant daily spending limits. I would love to be able to tell my bank that amazon is allowed access to up to $20/day or something of money until they get rate limited and I have to intervene.

Uh oh I better stop before I start advocating for the blockchain, haha

Checkout privacy.com you can set merchant daily, monthly or yearly spending limits.

Google cloud doesn’t allow those cards.

It's very ironical to see the word "privacy" in a thread about "serverless" stuff.

oh all the fanboy downvotes.

In Europe revolut has virtual cards with monthly spending limits. Cash App must have a similar feature in the US.

I believe you could accomplish this with Brex using virtual cards and per-card spending limits.

Are there any cloud products that have hard $$$ caps? Even non-big 3. Any at all?

Seems like this is a huge dealbreaker. You cannot be expected to perfectly audit your side project for security. Suppose someone finds a remote code execution and starts mining Monero on it. Or someone just points a botnet at it. You could be on the hook for an unlimited amount of money.

Might as well just install Kubernetes on a monthly-fee VPS and pretend you're using GCP.

Yes, Azure lets you set a per-Subscription spending limit.

It boggles the mind that AWS and GCP don't offer a payment cap - for that reason alone, I wouldn't go near them for side projects.

It would be nice to have three caps:

1. notify me so I can ok continued service 2. start throttling usage 3. turn off if #1 has not responded and set new limits etc.

It's cheap insurance to create an LLC to protect yourself from extreme situations for this kind of thing. Worst case the LLC goes defunct, but you won't be personally liable.

https://www.nolo.com/legal-encyclopedia/limited-liability-pr...

EDIT: This comment is based on the assumption that the parent was trying to start a side business. I'm certainly not advocating fraud.

wut? are you suggesting that if there's a cost overrun you just shutdown the LLC? that sounds like a loop that couldn't possibly be true? when you sign TOS you agree to be responsible for service fees. if being an LLC indemnifies against these sorts of charges it would also indemnify you against other legitimate fees? take out a business credit card and cash advance and then close the LLC? free money!

It basically does work like that, however the law is almost never that cut and dry. The situation you're describing would have the person benefiting from the money as being personally liable because that person presumably would be taking the cash advance for self-interest instead of taking the loan in the interest of the business.

more or less, yes. bankruptcy courts vary by location. the whole point of a 'limited liability corporation' is you not being personally liable.

BUT if you have 'malicious intent' -- meaning you build an llc on purpose to overrun costs and get free stuff, you CAN and WILL be sued and face penalties. cause that's just cut-and-dry fraud.

Companies are doing this all the time and not prosecuted (or prosecuted and not found guilty); they just do not make it obvious. So the ‘will’ is not that cut and dry really; even in countries where bankruptcies are not normal and frowned upon (like NL), there are many companies only created to spend money and killed off when it runs out. On paper they look like real businesses. Also it is not unusual to put employees in a separate llc and kill that when the money runs out; again that happens a lot, even with the intent of doing that. As long as it looks good on the outside, it works. I cannot stand it personally but there are not many solutions to resolve it; you cannot read the minds of the founders to get their real intentions.

Yes. This is exactly what the “limited liability” in LLC means. You are not personally liable, only the business is.

There are nuances to this, obviously fraud is fraud, regardless of using a shell company to perpetrate it, and a new business is unlikely to get a credit limit large enough to be a worthwhile avenue for fraud.

Banks often ask for personal guarantees from ltd owners for precisely this reason. I've just skimmed the T&Cs and I can't see any indemnity clauses, but that document's massive and I'm just some guy, so who knows.

(Incidentally, it's fairly common to structure a pair of ltds with the assets in one and the liabilities in another. If the whole thing crashes and burns, at least the IP doesn't go with it).

Which is exactly why banks require 2+ years of business history, collateral, or a personal guarantee. They will also review your financials when you apply for additional credit facilities.

As someone who used to work as an attorney in a past career, there's a policy called "piercing the corporate veil" to handle cases like this. Basically, if the only reason an LLC exists is to protect the owner from consequences of these sorts of shenanigans, the courts can pretend like the LLC doesn't exists for purposes of financial responsibility.

Note that this is a vague, general answer and most certainly does NOT constitute legal advice.

Good luck telling the judge your single person LLC with no investors shouldn't be pierced to pay debts incurred by its only owner and only employee.

Works fine if you didn’t co-mingle personal usage of the business accounts.

This VERY heavily depends on your state and I'm sure a lot of other factors. Years ago when I set up my single-member LLC my lawyer was quick to point out a thousand ways the LLC would be useless in court. Mingling bank accounts was only one of them.

The IRS, for what it's worth, treats a single-member LLC as a sole proprietorship and all losses/gains are considered personal income.

As always, check your state laws and consult a real, in-person lawyer.

A “cap” feature isn’t as easy to implement as you’d think. What happens when you hit the cap? Does it start shutting down instances? If yes, which ones and in what order? And don’t say “I don’t care” because you do care—you are allowing your provider to basically cause an outage in your service.

If you do allow some kind of ordered shutdown of service, what is the UX going to look like? What is the API going to look like?

Having a cap is not an easy feature to design or implement. And on a backlog, it is going to be a “multi quarter, low impact” item—meaning it will never get built.

Why not allow people to set a cap that disables everything set to that billing method once it's been reached? That seems to be the use case that people are saying is missing from cloud offerings.

Yeah after I posted I realized that once you hit your cap, that’s it you are done.

Still, I bet there is a lot of companies that might be more upset with the provider shutting everything down even if they had set up a cap.

I imagine most of those companies can afford to have some overages built into their caps. They should also be able to afford the developer time to create fallback behavior, and graceful degradation of services.

Things like storage (disks or S3) continue to incur costs even if the instance is shutdown. Should they just delete your data too?

What is the API going to look like?

Maybe just a simply array with the list of services to be shut down in the order provided? This should be enough for hobby and smaller projects, bigger projects will have plans/people in place anyway.

It is not that they can't do it - we are talking about a company with insane technical ability and resources. It is just that they don't want to do it.

What would you like to do with your VM disks and/or object storage when you hit the cap? Delete them?

Do-able today with GCP Billing

I thought GCP is alerts not capping?

The 'official' method is programmatically DIY!

https://cloud.google.com/billing/docs/how-to/notify#cap_disa...

haha

>Warning: This example removes billing from your project, shutting down all resources.

this is the digital equivalent of literally pulling the plug. i wonder how long it takes for GCP to register that you've pulled billing though? in some cases i'm sure even milliseconds could make a large difference.

Damn that's ugly.

I'm luckily still on Azure's MSDN tier...which is hardcapped to 150 bucks.

This is ridiculous. I'm guessing a huge revenue source is going after businesses for long-tail billing spikes.

Not only do I have to worry about bad code causing a huge bill, I also have to worry about the quality of code (or config) that emergency stops the billing.

I wonder if you can sign up with a pre-paid credit card and use a fake name.

I wonder how friendly the UI is to accomplish that

It's pretty complicated to do that on every services, imagine for any service they would have to call an API to get the $$ remaining , compute before scaling / creation if you have enough ect ...

If you wanted to be that precise, sure. But something like an hourly ‘calculate what I owe’ and compare it to a cap is not a big stretch of the imagination, and would suffice for many use cases.

A ml app could scale massively in an hour.

Then don’t get infinitely scaling services and get a Vultr instance, where your costs are fixed.

This makes me anxious as well.

Reply on news.ycombinator.com