Hacker News Reader

Comment by thomaslangston | Hacker News Reader

My concern is what it always been for AWS, no first party support for limiting my bill. The best effort from AWS is just an email notification when they "think" my bill is over a limit. If I can't limit my risk to a surprise bill of thousands of dollars after missing a midnight email, I still can't use their service at any usage based price other than free.

View on HN

You actually can do this yourself quite readily, through use of CloudWatch billing metrics (which can be broken down by individual service). Define some alarms based on billing thresholds, and perform automated scaling actions in response.

AWS can't exactly do this for you, as explained by previous commenters, but they do give you the basic features you'd need for implementing it.

There is a newrelic plugin that does this. It can graph your costs and raise alerts.

AWS Costs & Usage Analytics http://www.newvem.com/new-relic/

If you can get something to monitor your billing rate, you might be able to get PagerDuty to call you if your billing rate goes over x per hour.

While not the best solution, you can call Amazon, speak to someone to have your "surprise" bill mitigated.

This is 1st hand information from a AWS Rep I once dated.

Small yes. Large bill wasn't so easy. Once I had an excess $40 added to my bill after a network attack. I was able to down to $10 after a few emails back and forth. Another time I had something low like $10 and they credit me right away without problem. Depends on how large your bill...

You live in a very different world from me if a "large bill" is an extra $40. For even a small company like who I work for, any fluctuations in our AWS bill under $100 would probably go unnoticed. Our total infrastructure cost (of which AWS is a small part) is in 5 digits.

Surely your dev team laptops total 5 digits already.

Personal website of course it would be a big sum...

According to her, she would routinely have one-time massive fee's credited ($1000+).

I'd assume since it's a spike in costs, Amazon has more interest in keeping you as a customer, at your much lower typical usage, than to have you abandon, even if you have a huge bill.

Obviously if this was a typical problem, you would need to rethink your architecture and infrastructure needs.

That is so wrong it's not funny. Call? Isn't our use of computers predicated on the notion that we don't want to do manual labor?

Moving your mouth muscles isn't significantly more manual then moving the finger muscles you used to type this comment.

In either case, expending a few neural cycles prior to the action would be beneficial.

Personally, I find it hilarious that a simple verbal interaction with another human being is now categorized as "manual labor".

Please, point out to me the system that adequately resolves unusual billing situations without human intervention on both sides. I'd like to know what has set your bar so high that it prompted this level of entitled commentary.

Sat next to a businessperson on a flight to Vegas, and he was opining that "schmoozing" is becoming a lost art.

I'm all for being able to initiate billing inquiries online--in fact, I'm ditching Bank of America because I can't dispute charges on line.

But in Amazon's case, this isn't a dispute, rather this is asking them to forgive a (hopefully) one time transgression, which is much easier to do by speaking to someone, than filling out a form.

I find it similar to haggling for a temporary reprieve in my fantastically high cable bill.

My main gig revolves around bill auditing. You'd be amazed at the number of Fortune 500's that are supposedly driven by technology that have very manual processes around billing.

I used to do a billing system that processed about $15 million a year. To my horror, I realized when I joined that all chargeback processing involved someone in accounting going through the huge envelopes of chargeback claims + photocopied statements that they received in order to determine what to challenge. The online payment processor they used at the time didn't have any option for presenting the documentation online....

I work at one. I know, sadly.

I think thought that it's a failure of technologists to properly address the challenges of billing.

Well, you can do a ticket, too.

You can set up billing alerts which sends you alerts if your bill exceeds a value (set by you)

This is very hard for them to do.

For example, what if you have set up a bunch of auto-scaling instances to handle heavy load. During the middle of the night, your application suddenly goes viral in Indonesia or Brazil or some other large country in a different timezone from yours.

Your bill goes over its threshold - what does Amazon do? Do they terminate some of your instances (potentially losing production data!)?

The problem is that the exact times that "intelligent" bill-limiting features are likely to be used are the exact instances in which they could have absolutely disastrous impact if they make a mistake[0]

I haven't experienced this myself, but I've heard Amazon is actually rather lenient when it comes to waiving fees for clear mistakes (accidentally provisioning an expensive instance that you didn't mean to) - their customer support is supposed to be very good, though I have not had to use this myself.

[0] By "mistake", I mean that they are following your instructions correctly, but your instructions happen to be incorrect for handling this situation.

Yea any intelligent bill limiting needs to be done with context into what each instance is doing. It's possible for them to implement something that can handle that, using tags and such, but I would rather they spent that time improving reliability infrastructure.

It's not terribly difficult to implement one's own bill limiting system using the API.

It seems impossible for a single solution here. One business would rather see instances go down, another might need ledger logs no matter what and would rather turn down customers at the door, etc.,

The right way to do it (currently) is to model the costs yourself and tie it in with your operational automation, then determine what your best course of action would be during a shitstorm. Generally, I would wager the best option is to replace your front page with a static html "overwhelming response" page, maybe with some light vector graphics.

> Do they terminate some of your instances (potentially losing production data!)?

If you have opted in for the "don't let me spend more than $x," then yes.

I'd say the more intelligent way to head any of this off is to create billing alerts which you can now do and set it to page / email / post to an endpoint when you reach $X or X% of whatever worries you.

Create tiers of things that email / SMS additional people as the numbers go up if you want to be very careful etc...

I'd second that! It's not complicated, especially for smaller entities. "I'll use your services, but don't go over ths amount; without authorization." If my wiz, bang app, website has that much potential--I'll get the money to you. I don't want to wake up to a big bill. To be honest, if they were very clear about pricing, and stop/loss orders; I think more people would sign up. It's about honesty, and a minimizing the variables. Oh, and fire any MBA advisors; they think they are slick, but your average dude us starting to look at the fine print, and has become very sensitive to being taken advantage of. By the way, off topic, but relevant. I looked into esurance. Very good price. Then I realized the premium was only fir six months. I always wondered how they could afford all those rediculous commercials.

6 months is industry-standard quoting for auto insurance -- not sure why, but it is. What product are you talking about?

I accidentally setup EC2's in another Region (and effectively "forgot" about them. Woke up with a 1.2k invoice + $250 more in charges for March.

Amazon voided all charges without question. Their level of customer support is off the charts. But AWS would never be my first pick to start up something. Your costs will be way way lower when going dedicated. Obviously you should be able to scale, but I think startups generally overestimate their success when it comes to that, and wasting a lot of money for things they have no use for yet.

You could have your AWS deployment strategies ready for when the need would arise..

Amazon also has a startup program where you can get like $7k or similar free AWS credit when you register. They do some basic checks, but it's pretty easy to get IIRC.

EC2 pricing is more in line with other vendors. S3 is silly cheap.

If you're able to set up a service that risks a high bill, but cannot set up something that turns an email into some other action (anything from a text message to automatically shutting down your service), I don't think the limitations of AWS are the problem...

(Having said that, AWS could really do a lot better with it's notification targets than just email.)

Any service risks a high bill. Any object hosted on S3 could be the target of a DDoS attack, for example.

Hence the reason why you should be signing your S3 URL's.

Reply on news.ycombinator.com