For a company that supposedly puts the customer first, this is appalling.
There are a number of resource types that, simply by existing, will accrue costs. A lot of them, actually. On AWS that includes things like running EC2 instances, EBS volumes, RDS databases and backups, DynamoDB tables, data in S3 buckets, and more. The question is what should happen to these resources upon hitting a billing ceiling?
Should EC2 instances be terminated (which deletes all data on them), DynamoDB tables deleted, S3 data erased, RDS databases deleted? If that was the behavior, it would be an extremely dangerous feature to enable, and could lead to catastrophically bad customer experiences. This is a nonstarter for any serious user.
Conversely, if you expect those resources to continue to exist and continue operating, then that's basically expecting the cloud provider to pay your bill. The provider will then have to recoup those costs from other customers somehow, and so this option sets poor incentives and isn't fair to others. If you expect your account to remain open the following month, you'd have to settle the bill, and we're back to square one.
AWS gives people tools to tackle this problem, such as billing alerts. These can notify you over SMS, email, or programmatically when you hit an "$X this month" billing threshold, and then you can decide what to do. Since these events can be processed programmatically, it's possible to build a system that will automatically take whatever action you'd like AWS to take, such as shutting things down or deleting resources.
If you think all of this through, it's really hard to come up with an approach to billing limits that's fair and a good experience, so I think it's reasonable for cloud providers to give billing threshold alerts while leaving the choice of what to do in the hands of the customer.
Citation Required
So maybe a couple of EC2 instances go down, but you pay for and keep S3, Dynamo, etc. At least enough to salvage or implement a contingency. You'd still owe Amazon the money.
It's tempting to wonder why Amazon would incur that risk, but it is a risk already inherent to their post-pay model, and it serves as good faith mitigation to the runaway cost risk that is currently borne by the customer.
Not perfect, but maybe a compromise.
Let's take a simplistic example and say you're paying per gigabyte. You decide you're willing to pay up to $X, and Amazon tells you ahead of time how much your $X will buy you, and you accept.
One type of customer will be using that storage to store priceless customer photos. Even if the customer ends up deleting the photos, it has to be your customer who makes that decision - not you, and not Amazon. You tell Amazon that you'd like an alarm at $X-$Y, but that if you hit $X, keep going, at least until you hit $X+$Z.
Another type of customer will be using it to store a cache copy (for quicker retrieval) of data backed up in a data warehouse somewhere. You tell Amazon that you'd like a policy which automatically deletes all the oldest data, to guarantee to stay under the limit.
Yet another type of customer would rather keep their old data and just return an error code to the user for stuffing too much new data into too little storage, so basically, guarantee to stay under the limit, and guarantee never to delete data.
You can't solve billing until you communicate with your customers and ask what they want.
So lets for a moment assume you talked to a large cohort of customers, and found a bunch of "types" including those three you list and many many more (inevitably, at AWS's scale).
You then need to make some business decisions about which of those "types" are most important to you, and which are way less profitable to spend time addressing.
So of course you solve the big pain points for your customers spending tens or hundreds of thousands of dollars per month before you prioritise the customers worried abou going over a tens or hundreds of dollars a month budget.
What would that solution look like? It'd have ways for customers with hundreds or thousands of services (virtual servers, databases, storage, etc) to make all their own decisions about alarms, alerts, cost ceilings - and tools to let them decide how to respond to costs, how to manage their data availability, how to manage capacity, when to shut down services or limit scaling, what can and cannot be deleted from storage. It would also 100% need to allow for practically unbounded capacity/costs for customers who need that (Think AliExpress on their "Single's Day" event where they processed $1 billion in sales in 5 minutes.) All this would need - for the $100k+/month customers - to be machine drivable and automateable, with extensive monitoring and reliable alerting mechanisms - and the ability to build as much reliability and availability into the alerting/reporting/monitoring system and the automated provisioning and deprovisioning systems as each customer needs.
And at least to a first approximation - we've just invented 70% of the AWS ecosystem.
You might think Amazon don't cater to people who want hard $5 or $70 per month upper limits on their spending. You're _mostly_ right. There are many other people playing in that space, and it's _clearly_ not a high priority for Amazon to complete for the pennies a month available in the race-to-the-bottom webhosting that people like GoDaddy sell for $12/year.
The thing to think about is - "who does Amazon consider to be 'their customers'?". I think you'll find for the accounts spending 7 figures a year with AWS - billing _is_ "solved". The rest of us are on the loss-leader path (quite literally for the "free tier" accounts) - because Amazon only need to turn a few tenths or hundredths of a percent of "little accounts" into "their customers" for it all to work out as spectacularly profitably as it is doing right now.
Except that that's what this announcement is.
Which makes me think this may be AZON's fix to runaway billing - if you don't have the resources to pay for mistakes[1], stay in the per-month kiddie pool and don't play with the heavy machinery.
[1] I started to add, "or trust yourself not to make them", but that's silly, because mistakes will happen.
Let's assume, based on the evidence at hand, that Amazon is rolling out Amazon Lightsail, and that as such, they're willing to do work (create business plans and write software) to court the $5/month market. In that case, it's a relevant comment for people to write "I can afford $5/month, or even $20, but I can't afford unlimited liability, even with what I know about AWS customer service, so I cannot use this product." It's relevant because it suggests that there's anxiety that is preventing uptake, which can be solved by a combination of writing software and internally committing themselves to eat the loss if the software is imperfect (as others have said, stopping service actually-on-time is actually harder than it sounds, but the provider can always just eat the loss, invisibly to the seller).
Your (probably-correct) observation that Amazon doesn't really care about the penny-ante user's money (in the short term) is beside the point.
Most companies will hold onto your data for a time, then delete it afterwards.
This doesn't smell like technical concerns to me. It smells like sneaky Amazon-wants-to-make-more-money concerns.
(<snarky> What's a gallon of milk on the shelf really cost Walmart? And how much of it is opportunity cost? If I usually buy 2 gallons a week - why can't I keep taking home a gallon every few days for a month or so after I stop paying, then cut me off afterwards? Sounds like a sneaky Walmart-wants-to-make-more-money concern.)
If only Walmart would have a process in place to notice that I was ordering a spectacular and unusual amount of milk and save us all the trouble.
So my local Walmart has a Netflix guy who gets 1000 trailers of milk twice a day, and the Dropbox and Yelp guys get a few hundred trailers a week each - and I know these guys from when I see them at the other Walmart in the next town over buying the same sort of amounts there as well. There's people like the Obama campaign who we'd never seen before who fairly quickly ramped up from a gallon a day to a pallet a day, then jumped straight to 50 trailerloads a week for a six months, then stopped buying milk completely one day.
What's considered "normal", "unusual", or "spectacular" - and to whom?
Plenty of companies operate like that, and e.g. require purchase order ids and accompanying maximum spends issued for any expense over X, where X can be very low. I've worked for companies where it was 0 - every expense, no matter how low, needed prior approval from the CEO or finance director. Not just tiny companies either - one of the strictest such policies I've dealt with was with a company of more than a hundred employees.
Amazon AWS's "important customers" are not "fallible human beings" who plan to keep their monthly spend under $100. They'd perfectly happily inconvenience thousands of those users in favour of their customers who _do_ need solar system scalability.b(And, to their credit, there's an abundance of stories around of people on typically 2 digit monthly spends who screw up and get a 4 digit bill shock - which Amazon reverse when called up and pleaded with.)
So they built their thing as "default unlimited". Because of course you would in their position - follow the money. When Netfix wants 10,000 more servers - they want it to "just work", not have them need to call support or uncheck some "cost safety" checkbox.
If you need "default cheap", AWS isn't the right tool for you. You can 100% build "default cheap" platforms on AWS if you've got the time/desire (well, down in the "I can ensure I don't go over ~$100/month - it's not real easy to configure AWS to keep costs down in the $5/month class - the monitoring and response system needs about twice that to keep running reliably).
I sometimes don't think people (especially peope who "grew up" in their dev career with "the cloud") understand just what an amazing tool AWS is - and the fact that they make it available to people like me for hobby projects or half-arsed prototype ideas still amazes me. I remember flying halfway round the world with a stack of several hundred meg hard drives in my carry on - catching a cab from the airport to PAIX so I could open up the servers we owned, and add in the drives with photos of 60,000 hotels and a hardened and tested OS upgrade. Buying those 4 servers and the identical local setup for dev/testing, getting them installed at PAIX, and flying from Sydney to California to upgrade them was probably $30+ thousand bucks and 3 months calendar time. Now I can do all that and more with one Ansible script from my laptop - or by pointing and clicking their web interface.
AWS is an _amazing_ tool - talk to some grey-beards about it some time if you don't remember how it used to get done.But the old saying holds: "With great power comes great responsibility." If you don't want to accept the responsibility, use a tool with less power. Don't for a minute think Amazon are going to put a "Ensure I don't spend as much money with AWS as I might otherwise" option in there - if there's _any_ chance of it meaning a deep-pocketed customer _ever_ gets a false positive denial from it. (WHich, now I think about it - makes this new Lightsail thing make so much more sense...)
Also how our are analogies alike? Milk is a consumable, data is information. Completely different usage pattern.
Finally, every internet service provider I've ever used that held data for some reason granted me a grace period, even if it was never officially stated. Sometimes you just have to ask nicely
When a customer's ceiling is reached, their mix of services goes into limp mode. Things slow down, degrade, maybe become unavailable, depending on each service's "freeze model". Alarms ring. SMS messages are sent to emergency phone numbers. The customer is given a description of the problem and an opportunity to solve it -- raise the cap or cut services.
So wouldn't this cost Amazon money? Sure, but that's a cost of doing business. And as others in the thread have pointed out, the actual costs to Amazon are surely much lower than the "loss" they're incurring by not unquestioningly billing the customer. Especially since Amazon often refunds large surprise bills anyway.
If this were the official policy -- no dickering required -- there's a definite cohort of risk- and uncertainty-averse customers who would be willing to start using Amazon (or switch back).
That's what stopping instances _is_ already. You don't get charged for stopped instances which is a defining feature of Amazon's cloud. Very few providers actually offer this. Most just charge away for the compute even if the instances are powered off, Azure being one exception.
This whole "spin up compute and get charged a minimal amount when not in usage, but keep your working environment" model was pioneered by Amazon.
> So wouldn't this cost Amazon money? Sure, but that's a cost of doing business.
Why would Amazon spend a bunch of money, so that they can charge customers _less_ money, in order to keep customers who are cheapskates, and/or won't take the time to learn the platform properly?
I've seen engineering teams hand out accounts to support teams for testing, and since the resources are not under the purview of the dev team things go unnoticed until someone gets the bill. Arguably there are better ways to handle these requirements, but it'd be nice if you could force people down the path of setting billing alerts because these individuals don't always realize that they are spending money.
But, they don't don't give us the choice. I need to keep an eye every moment of every day for an alarm, as hundreds or thousands of dollars rack up. That's the ONE THING I DON'T WANT. I'd take anything else (delete my data, lock everything, whatever) over charging me money I can't afford to pay.
I think it would be reasonable to put everything into a no access / deep freeze mode, until I pay up and choose to unfreeze. Would it cost Amazon that much to just keep my data locked for a couple of weeks while I sort out my storage? I'd even be happy for a reserved $100 or so to pay for keeping the storage going.
You know you can make a machine do that for you - right?
In fact all the tools Amazon would use to do this are available to you right now. Cloudwatch, SNS, and Lambda are 98% likely to be all you need - apart from the time to get it set up to do whatever you think is "the right thing".
This seems like the kind of thing you really want to get right, and it will be (I imagine) hard to get right. If it was easy, I would expect some company to offer it (along with, of course, a guarantee that if they mess it up, they will pay my bill).
Nobody rings up Caterpillar and complains about the costs of leasing/running/maintaining a D9 'dozer if they're doing jobs that only need a shovel and a wheelbarrow.
Tools for the job. AWS might not be the tool you need. Or might not be the tool you need _yet_.
But.
There's nothing "unexpected" or "unagreed beforehand" about Amazon's pricing or costs either. You order a medium EC2 instance and we all know exactly what the bill per hour will be.
There's nothing unexpected or un agreed beforehand about the ordering/provisioning process. You ask AWS to start one, they'll start one. You tell them to stop it, they'll stop it. You get charged the known agreed upon rate for the hours you run it. You ask for 10, you get 10. There's even checks in place - the first time you ask for 50, you hit a limit which you need to speak to them to get raised before you can get a larger than previously seem bill.
Same with your earthmoving gear. You ring up for prices and they'll say "$200/day for a bobcat, $2500/day for a D9 - includes free delivery in The Bay Area!"
If you need one bobcat for one day at 10 Infinite Loop, Cupertino - and click their web order form and say you want 10 D9s for one day at 1 Infinite Loop, Cupertino (and happily click thru all the never-read the web interface confirmations) - you should 100% expect to get a bill for $25k, as well as dealing with clearing up after parking 10 'dozers in Apple's parking lot.
This is not "unexpected". From the vendor's perspective $25k is not "massive". You knew and agreed to the prices and had every opportunity to calculate what your bill was going to be.
If you were only expecting a $200 bill - that's kinda on you. The earthmoving guy has heaps of other customers who spend many times that every single week - and they all started out as some guy who ordered a $200 bobcat or $25k's work of D9's as a one off. You are just another sale and another prospect in the top of the MRR funnel for him.
(Note: See holidayhole.com for a contemporary example of an unbounded earthmoving bill! ;-) )
The problem is someone putting up your hobby website on reddit when it's 2 in the morning your time, and you wake up the next day with a $10,000 bill.
Even heavy equipment rentals can result in large unexpected bills if you don't pay attention to what you're doing.
I need "reddit / DDos insurance"
You know exactly how much a paused EC2 instance charges you. The ceiling implementation could say, if the total amount charged so far this month, plus the cost of pausing the instance for the rest of the month, exceeds the ceiling, pause it now. So there's no data loss; the worst case is the customer's service is offline for the remainder of the month (or until they approve adding more money). At some point less than this number, start sending angry alerts. But you still have a hard cap that doesn't lose data.
It's not what a serious production user wants, but it's exactly what someone experimenting with AWS wants, either a running service that's looking at a cloud migration, or a new project/startup that hasn't launched yet.
Granted, for a big company, that amount may be so big it's unrealistic to ever hit it.
No one running a real business on AWS wants a hard ceiling instead of billing alerts and service by service throttling. Which Amazon has.
So, this is just the nuclear option for people's pet projects. It's not a bad thing to have but I wouldn't expect it to operate any differently than what would happen if you broke the TOS and they suspended your account.
Are you sure? I'd imagine many startups would rather take a few hours of downtime over billed thousands erroneously. The latter could easily mean the end of the company but the former, when you are just striking out is not the end of the world by far.
That's absurd. Of course there are businesses that want hard ceilings. Perhaps not on their production website[1], but on clusters handed over to engineers and whatnot for projects, experimentation, etc.? I've seen these things lay around for months before they were noticed.
[1] Maybe you don't consider startups 'real' enough, but I can totally imagine early stage startups wanting limits on their prod website, too. You can't save CPU cycles for later consumption.
I know startups that I could bankrupt with a few lines of code and a ~$60 server somewhere long before they'd be able to react to a billing alert if it wasn't for AWS being reasonably good about forgiving unexpected costs.
I'm not so sure no one running a "real business" would like a harder ceiling to avoid being at the mercy of how charitable AWS feels in those kinds of situations, or when a developer messes up a loop condition, or similar.
Perhaps not a 100% "stop everything costing money" option that'd involve deleting everything, but yes, some risks are existential enough that you want someone to figuratively pull the power plug out of your server on a seconds notice if you have the option.
If you can't afford downtime you probably can afford to wait for the alert and choose your own mitigation strategy. A system that can't tolerate downtime probably has an on-call rotation and these triggers ought to be reasonably fast.
If you can't react or can't afford to react, you probably can afford some downtime / data loss.
So the system doesn't need to have granular user defined controls. Just two modes. That was my point.
I think I triggered people with the phrase "real business" and I apologize for that.
Only a tiny fraction of businesses can't afford downtime. A lot of businesses claim they can't afford downtime, yet don't insure against it, and don't invest enough in high availability to be able to reasonably claim they've put in a decent effort to avoid it.
In most cases I've seen of businesses that claim they "can't afford downtime", they quickly balk if you present them with estimates of what it'd cost to even bring them to four or five nines of availability.
> A system that can't tolerate downtime probably has an on-call rotation and these triggers ought to be reasonably fast.
A lot of such systems can still run up large enough costs quickly enough that it's a major problem.
> If you can't react or can't afford to react, you probably can afford some downtime / data loss.
I'd say it is the opposite: Those who can afford to react are generally those with deep enough pockets to be able to weather an unexpected large bill best. Those who can't afford to react are often those in the worst position to handle both the unexpected bill and the downtime / data loss. But of the two, the potential magnitude of the loss caused by downtime is often far better bounded than the potential loss from a crazily high bill.
On the other hand, based on near-universal industry practice, there doesn't seem to be a huge demand for this. I suspect it may be better for everyone concerned to have heavy-duty users control their costs in various ways and for Amazon to refund money when things go haywire without bringing someone's service down.
Not saying Jevon's Paradox wouldn't kick in, but the friction of convincing businesses to work on tools to allow their customers to spend _less_ money is high.
One of the most amazing feats Amazon has pulled off is to convince people that AWS is cheap. They're cheap in the way that Apple are: Only if you need a feature-set (or name recognition..) that excludes the vast majority of the competitors from consideration. If/when you truly need that, then they're the right choice. There are plenty of valid reason to pick AWS.
But they're very rarely the cheap choice.
Yes, but that's a false comparison. It's cheaper to rent dedicated servers at any of several dozens large hosting providers than it is to use EC2 or S3, for example. For most people it's cheaper to rent racks and lease servers too (but depending on your location, renting dedicated servers somewhere else might be cheaper - e.g. racks in London are too expensive to compete with renting servers from Hetzner most of the time, for example).
It's extremely rare, and generally requires very specific needs, that AWS comes out cheap enough to even be witting batting range of dedicates solutions when I price out systems.
When clients pick AWS, it's so far never been because it's been cheap, but because they sometimes value the reputation or value the feature set, and that's a perfectly fine reason to pick AWS.
The point isn't that people shouldn't use AWS, but that if people thing AWS is cheap, in my experience it means they usually haven't costed out alternatives.
It's an amazing testament to the brand building and marketing department of Amazon more than anything else.
E.g. my object storage costs are 1/3 of AWS. My bandwidth costs are 1/50th or so of AWS prices.
There are valid reasons to use AWS depending on what exactly you do, but it's extremely rare for price to be one of them.
This is one of the fundamental things that make any sort of market work. If it's not safe to participate, people won't.
The real economic term for this is elastic demand (specifically, relatively elastic demand). For example, microprocessor cost reductions make new applications possible, thus demand increased so much that the total amount spent on microprocessors went up for decades. Example of inelastic demand is radial tires. They last four times as long as bias ply tires. But since this didn't cause people to drive four times further, the tire industry collapsed on the introduction of radial tires.
Does anyone know an example of an actual paradox? I've never found one, and I'm curious if they really exist.
Jevons's Paradox is about demand increasing for a resource when it becomes more efficient to use, e.g., someone invents an engine which can go twice as far with the same amount of fuel but instead of halving the demand for fuel the demand actually increases.
If I recall, elasticity of demand has to do with the relationship to supply. A very inelastic demand will cause people to consume the same rate no matter how much _supply_ is available. It doesn't have to do with the efficiency at which the resource is consumed like stated above. It's a subtle difference but I think they're actually quite distinct concepts.
Actual paradoxes are common. Just consider the classic: "This sentence is false".
As for your example, most sentences are neither true nor false. Nothing interesting has a probability of 0.000 or 1.000.
"This sentence is false" is clever use of language, may be interesting to sophomore philosophy students while smoking weed, but its not useful and there's nothing paradoxical about it.
> most sentences are neither true nor false. Nothing interesting has a probability of 0.000 or 1.000.
I'll start by observing that surely you're talking about propositions, not sentences, nor utterances. Or at least you ought to be.
But more significantly, I'll note that most propositions are either true or false (under a given interpretive framework), but that as epistemologically-unprivileged observers, we must assign empirical propositions probabilities that are higher than 0 and lower than 1. Propositions like "I am a fish" or "You hate meat" or "If Rosa hates meat then Alexis is a fish" are either true or false, under any given set of meanings for the constituent words (objects, predicates, etc). I'm curious what probability you think applies to propositions like "2 + 2 = 4" and "All triangles have 3 sides" and "All triangles have less than 11 sides". I think there are very many interesting propositions that differ from these only in degree of complexity (e.g. propositions about whether or not certain code, run on certain hardware, under certain enumerable assumptions about the runtime, will do certain things).
Based on your very strange claim that all interesting sentences have non-zero non-unity probability, perhaps you're saying that you find theorems uninteresting, and moreover are only interested in statements of empirical belief, such as "I put the odds of the sun failing to rise tomorrow lower than one in a billion." In that case, I cannot imagine what statement interest would qualify as a paradox, except perhaps insofar as some empirical statements of belief are "beyond strange".
"This sentence is false" is a paradox under pretty much everyone's notion of a paradox.
Those are great examples, thanks. All true, and there's nothing interesting about them.
And these things are exactly the sort of thing that "differ from [trivialities about triangles] only in degree of complexity".
Note that "all triangles have 3 sides" is probably an axiom, but "all triangles have less than 11 sides" is a trivial theorem.
If you need to dig this hard to find something interesting with a probability of 1, that's pretty good evidence that the vast majority of interesting statements are not of the true/false variety.
Although I don't find it interesting, I am open-minded enough to ... embrace.. the .. uh.. diversity of the world, that allows some people, to find that interesting.
The language that contains all Turing Machines that halt on all inputs is not decidable.
Or
e^(iy) = sin(y) + i * cos(y)
Are those uninteresting trivialities to you?
My hypothesis is that they don't really have it nailed down but given big margins they have they can afford to let you use more resources than you pay for in the end.
[1] https://azure.microsoft.com/en-us/pricing/spending-limits/
There are a few Azure services to which the spending limit does not apply, but as long as you know what they are then you can choose to use them on your own volition.
One time we had literally 1 million cloudwatch metrics get created because we were monitoring mongodb databases and a CPAN test was creating test DBs and not deleting them and we were not ignoring dbs created with names like test_* when creating the metrics.
Another time an outside developer committed a root credential on a public repo in github to a (basically) unused amazon account.
Both times they refunded the costs. Not sure if that was because we were paying tens of thousands a month though to get this service though!