My First AWS “Free Tier” Hosting Bill Was $990 | Hacker News Reader

My First AWS “Free Tier” Hosting Bill Was $990

blog.andrewray.me

As somebody who deals with AWS professionally on a daily basis, I will 100% admit that it can be confusing for people to figure out pricing and scale. I have to talk with engineers every week about design decisions and their cost impacts, and the implications of the things they want to run, and it's clear that AWS doesn't give enough guidance in the UI or documentation structured in a way that a layperson could find what they're looking for quickly and easily for that information.

That said? This article rubs me the wrong way. The suggestion that this problem is caused by "dark patterns" and Amazon being misleading about "pay as you go" screams FUD to me. The screenshots provided right in the article clearly show that what was being done wasn't eligible for free tier. And not understanding that launching a resource, regardless of whether or not you're interacting with it, is consuming the service which launches the resource is a problem with the user, not the marketing. Yes, the author admits that they didn't scrutinize, but that doesn't excuse the position of the article that AWS is somehow doing this to intentionally bilk people.

My favorite part is how half of the screenshots include a clearly marked "Free Usage Tier" option that OP did not select

https://blog.andrewray.me/content/images/2018/03/Screen-Shot...

Dark UI indeed.

https://blog.andrewray.me/content/images/2018/03/aws-rds-now...

So, let me get this straight: - OP ignores the obvious "This does not fit the free tier" warnings when setting up their app. - OP does not pay attention to any billing metrics or even bother to try and understand the pricing beforehand. - OP gets hit with a $990 bill. - Amazon gives all the money back, plus free credits. - OP complains.

AWS should have kept their money and OP should've learned their lesson proper.

The author of the post also calls out that the mistake was his, despite the occasional muttering about "dark UI patterns."

I spend a lot of time in AWS, and I have trained myself to be extra careful about reading the fine print when using the UI exactly for the reason the author describes.

The author calling out his own stupid mistakes elevates him in my eyes, not the reverse. Honesty and recognition of wrongdoing in oneself are important traits.

> The author of the post also calls out that the mistake was his, despite the occasional muttering about "dark UI patterns."

Yeah I saw that like, and lolled. The author does indeed admits his/her mistakes, but proceeds to kinda blame it on AWS anyway.

It's like like saying "look I am no racist but <insert some very bad racist phrase here>".

But, so, at what point, like how many people have to experience making this mistake, does it have to be for it to become AWSs problem?

Everyone makes mistakes, you can either help them not make these mistakes or do nothing and blame it on the users.

That's a terrible analogy.

AWS's UI is bad. The author's mistake was exacerbated by this, even if it wasn't directly caused by it.

The problem that I had AWS was that it felt hella confusing, flooding you with information about services they provide and their AWS-specific buzzwords, sort of drowning out a lot of the important info

If we are trading war stories:

I was playing around with some tutorial to learn something (probably something cool like programming your own robotic drone using functional erlang or whatever), pushed to github and went to sleep. Woke up a few short hours later and had lots of emails about the machines I was spinning up.

Checked and saw that my account had wracked up thousands of dollars overnight (I think 6-8 hours), and I started to shut down the machines.

I didn't get them all, there were more machines hidden, and the bills continued to pile up for another hour or two.

I contacted Amazon who shut it all down, and I reset my password.

Then I realized I had pushed my credentials to github (I should really put this under a pseudonym, but I was new to the whole thing and hadn't even looked into Amazon's authentication system. Obviously, billing credentials and sysadmin credentials should never be the same.) Someone had a scraper going that picked them up almost right away.

To Amazon's credit, they cancelled the charges within a few hours, and if memory serves the person investigating gave me a sympathetic but stern message.

I don't know who the credential-stealer was and what they were using it for, but I would guess crypto mining. I did some calculation at the time and I think they would have extracted about 1/3rd the value of my bill, but those were rough calculations.

Credentials on github is actually a fairly common cause for GDPR breach, not as common as people using auto-complete in their e-mail system, but it’s up there.

So you’re not as alone as you think, and these aren’t from people trying to learn something, it’s from big enterprise IT organisations.

Auto-completing e-mail adresses is a GDPR violation? Because you could iterate them and see all the contacts? Seriously?

Any exposure, intentional or accidental, of PII to a non-authorised person is a GDPR violation. An email address is PII as it's unique to that person.

Consider the Ashley Madison breach - there were websites that let you search for an email address and see if it was included. Even without the name or address of the person it was sufficient PII to cause damages (however 'deserved').

On public websites I would agree, but all our mail clients have auto-completion. So would we need to turn that off? Would probably disable half the company.

I don't know who Ashley Madison is but that sounds far beyond sensible protection. Given, auto-completion is restricted to employees plus some locally saved contacts. It is just the standard outlook-exchange setup.

You can set up what (groups) autocomplete as a admin on both outlook and g suite (probably other providers).

Otherwise it's only people you have been in contact with.

No, but sending all that personal information to the wrong person is, and it’s the most common GDPR violation in my country.

I really don't think this is a UX problem. It was pretty clear that there were two options, production-ready and free. If you want to be picky, I suppose you can be upset that RDS is just a couple of VMs that you can't run other things on, or to question how much performance benefit a certain number of provisioned IOPS gives you. I don't think that's a dark pattern so much as "we don't know what your workload looks like, you don't know what your workload looks like, so just provision a bunch of IOPS and hopefully we never speak again."

I am less surprised that the mental model fell apart. I guess a lot of people think cloud resources are something that is efficiently shared (consider S3, you pay per byte you store, store 0 bytes, pay $0). But that's actually a rare case, most of the time you are provisioning something for your exclusive use; if you have a database server it costs you the same whether it's doing 10000 transactions per second or sitting completely idle and never logged into.

(Incidentally, the true sharing model used to be popular. Shared hosting with no isolation between tenants predated AWS by a decade. You got a chunk of a computer and shared Apache, MySQL, and PHP with hundreds of other randoms. Very cheap!)

If using AWS for personal use the first step should always be to set up a billing alarm. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

It’s very helpful. You can still end up overspending but at least you get an email within a day letting you know what’s going on, which can solve a lot of the cost overruns by giving you a chance to act quickly and only get hit with 1/30 the monthly fee.

AWS Budgets is a much better tool for this. It’s simpler to set up, no futzing with CloudWatch, and also alerts you when forecasted usage will exceed a set amount, so you are much less likely to overspend.

https://aws.amazon.com/aws-cost-management/aws-budgets/

Why is this an extra step instead of something AWS always does?

As both services and permissions multiply, the user experience of AWS is getting worse and worse. How would you even know to setup CloudWatch if it’s your first time using the service?

By default, AWS already automatically notifies you when you exceed 85% of your free tier usage. The author did not get these alerts because he ignored the multiple warnings that he was using services not included in the free tier. Because he wasn’t using the free tier, he didn’t get alerts for the free tier.

And even when not using free tier, one of the first things AWS tells you to do when creating an account is set up billing alerts.

I’ll be the first person to line up and say that the AWS console and UI is atrocious, especially for hobby devs. But this wasn’t that. The author completely ignored the multiple warnings, got himself into a pickle, got it resolved, and is still complaining about it.

Why is it an extra step to request a wake up call when you get a hotel room? Just call every room at 6am.

Why is it an extra step to set a timer when you turn the oven on? Just set it to 60 minutes.

Why doesn't every toothbrush beep when you didn't brush long enough?

Why aren't the police keeping track of my children when they go outside?

Because personal responsibility and due diligence is hard! /s

Why not have a system of pre-paid service? Cut-off once the fund dries up. This will be ideal for a testing.

For many businesses, the prospect of being cut off as a result of unexpected demand would be a serious liability, so therefore this would have to be an option -- but if it is an option, the customer has to select between it or an alternative, which is exactly the problem in this case.

Free tier is even more ideal for testing than this proposal, but the only way you can make options foolproof is to have no options.

My girlfriend's first "Free Tier" hosting build was $5200 for ingesting a single document on Kendra. After nearly a month of working with the service team she was able to get the bill removed (even the service team did not know how to delete the app). Its insane that an overage charge on a "Free tier" service can be the price of a used car or multiple months of rent.

Well their free tier is a list of free amounts you can consume of various services. Azure and GCP require you to switch to a Pay as you go scheme if you want to spend money. But AWS from day one you can consume things that cost money.

They say you get 30 days of free usage but with a 5k a month cost, no way I'd risk it with my card. https://aws.amazon.com/kendra/pricing/

I'm sorry, but that's on him. Look at the first screenshot in "Default Configuration". It says so very clearly. Twice. He admitted he didn't care, this is what happens when you don't.

I wish AWS had hard spending limits. Azure have one - you spend over the set limit(probably per billing account?) and your services are suspended. Already saved my from unexpected bill this month.

I like this option. It reminds me of a similar issue (which eventually got native AWS support): S3 permissions.

Today, there's a "Block Public Access" button which basically says "I solemnly swear that I don't want anyone outside of my account to see this S3 bucket. Please don't put this bucket on the public internet, even if I screw up my bucket policy and/or ACLs"

The option is off by default, but it's easy to find, simple to understand, and doesn't force powerusers to give up control.

[0] https://aws.amazon.com/blogs/aws/amazon-s3-block-public-acce...

Just setup billing alarms. Spend is reported continuously. These people surprised by end of month bills just aren’t paying attention to all the data AWS shows you on your spend.

If they hard shut people down then people would be posting “AWS turned off my services and took my site down blah blah blah”

What about having choice? Hard limit and alert? I’m been bite by overspending accidentally on Azure (only ~30€ but still) so the hard cap is a real reassuring thing.

It's not quite that easy. It works for stateless services like transfers that can be just stop doing whatever they're doing. But presumably you don't want AWS to start deleting S3 buckets if a threshold is reached.

I actually tend to agree that, especially for hobbyist use, an automated hard cut off that cut out most further AWS service use would probably be desirable--even if some would (mis)use it in production environments and end up blowing up their site and complaining about it. I'm sympathetic to those who find the potentially open-ended nature of AWS billing to be bothersome. An alert is just an alert. There's no guarantee you'll be in a position to receive it and act on it in a timely manner.

Set up a lambda function that looks up billing. If billing reaches certain threshold, start shutting down resources. We do that and it works just fine.

This is why they make so much money. Flat rate hosting is the only way I’ll go.

They bank on smaller bills and hope not everyone calls.

I prefer Digital Ocean, at least I know what it will cost, and working with the product is straightforward. Even huge company like the one I work for gets stressed out over bills when we do anything new with AWS (our budgeting process is scary too). At least they usually will help us out when things go wrong given how much dinero we give them.

>at least I know what it will cost

Yep until you go over the traffic limit and they start charging you more

That's why you set a billing alarm.

FTA:

> To Amazon's credit, they removed the bills from my account and gave me free hosting credits to make up for it. Their support was swift and professional.

That's not unlike a lot of other companies --- if you complain they'll be nice to you, but otherwise they'll eat all they get.

Perhaps it sounds a bit odd, but in most business settings, I'd rather be paying double and have service like this than be paying half without that customer service.

Generous policies like these avoid headaches, and headaches are much more expensive than machines.

With a model like this- you're only getting your money's worth while you're regularly making mistakes. If everything is going smoothly, you're just paying double for no reason.

I dislike when the value proposition gets muddied like this.

I'm paying double for peace-of-mind.

Think of it in terms of expected costs.

* In one case, I pay $500 / month.

* In the other case, I pay $250 per month, and have a 25% chance of having a workplace conflict. A workplace conflict can cost tens of thousands of dollars.

Which is cheaper?

Thanks, I did read the article. I clarified my post with an additional point.

Since the very beginning AWS has been an adversary to their own customers.

Either you invested your time/money in deeply knowing their ins and outs ( And you're fine spending your life that way ) or you're just a cog inside someones else's big wallet and don't care.

If you're not a big corp or don't have VC money to burn, there are much better options than AWS. The feeling of not getting f"#$ed over every step of the way is priceless, Azure is barely any better.

Heh, my company hosts a very small service with < 100 users a month for very specific b2b purposes. We pay around 20$ a month for complete hosting, although AWS is constantly reminding me that I could spend around 50% of that if I do X, whatever that may be.

We don't pay for support tiers and I was extremely surprised that we got a response within ~6-8 hours to unlock their mail-service (SES). You have to do your homework to convince the support employee that you are not building the next spam-network. So they actually have to read all your antics.

It can be a solution for small business with lower traffic applications and after being surprised that they didn't just ignore my request I cannot say their service is bad.

I have another AWS-account, but that is unrelated to the one I use for my current company. Billing of all cloud services is intransparent and I can only believe them if they say I used n hours of CPU time. Don't even know how I would begin in calculating that. Still, their billing console is very helpful. I just ask myself why they put links to your requests for payment everywhere, but not to the actual tax-invoice. That one is ridiculously hidden.

I never had any training for AWS and I tend to skip reading documentation if it gets too boring. They are certainly expensive buttons, so my advice would be to use the credit card of your employer to check it out.

Amazon have entire teams of people who's job is to optimise customer's bills so that you end up paying less, and they're incredibly good at it. You need to become a big enough customer that they assign an account manager to you for that to become apparent without doing some digging, but I'm pretty sure if you open a support ticket asking about this they'll help you out.

Their job is to make you don't pay over the threshold where you would leave. They are obviously not on your side.

If Amazon wanted to solve these problems they would change the pricing structure, they are not stupid, they know exactly what they are doing since day one and it's working. The moment it stops working for them they will do something about it.

From https://forums.aws.amazon.com/thread.jspa?threadID=58127

> The community has requested this many times and you promised the feature yet stalled it for many years now.

People in this thread say they started promising this in 2006.

And the solution to this and all the other "free" SaaS conundrums is the same as it always has been: run it on the server-under-the-stairs or, if need be, on a flat-rate hosting platform. You might not be able to tick off all the buzzwords but you know up-front what you are getting in to and you're immune to all the problems listed in this thread. You might be trading them for some other problems but those are far less likely to break the bank.

I completely wasted $500 of free google cloud credit from having a 2 core Windows VM "idling" for a couple of days... thank god I checked the balance

I'm dated on my cloud vendor costs, but a 2core for a day is 500? Most of the cost the license? Windows server?

That's why I still rent VMs for a fixed price

I have to admit, AWS isn't easy.

I did two associate cers, just because, and after getting them I had the impression that I vastly underestimated the difficulty of using AWS.

If you really know what you're doing, AWS is probably much better and often even cheaper than everything else, but most people simply can't put in the time.

tl;dr: Author opted for not actually using "Free Tier" and ignored all the (subtile) warnings.

> I didn't know what "Multi-AZ Deployment" nor "Provisioned IOPS Storage" were, nor did I care.

No. RTFM and look up all the terms you're not familiar with.

> Production is production, right?

Right.

tl;dr: some guy didn't pay (pun intended) attention to pricing and the scope of the free tier and got a huge bill.

nothing new under the sun, once again.

TL;DR: Misleading title. Amazon service came through.

* Author clicked through Amazon UI/UX, and ended up with a huge bill.

* The UI/UX was confusing and poorly designed -- at no point was he shown he'd pay anything, let alone a lot.

* He was refunded the money AND given credits to make up for the hassle.

This is one of my key frustrations with Amazon. (1) I'd like services like RDS or similar on a pay-as-you-go fashion, rather than based on spun-up servers. I'd like SQL-as-a-service where I pay for actual storage and operations (without dedicated machines). (2) I'd like to understand pricing up-front, and be able to track what I'm paying.

Still, beats everything else.

>(1) I'd like services like RDS or similar on a pay-as-you-go fashion, rather than based on spun-up servers. I'd like SQL-as-a-service where I pay for actual storage and operations (without dedicated machines).

Seems a somewhat hard problem due to latency constraints and advantages of memory caches. For big data you have AWS Athena and Google Big Query. There's also auto-scaling if you're large enough of either read replicas (AWS Aurora) or the whole thing (Google Spanner).

>and be able to track what I'm paying.

Amazon does provide a nice billing dashboard which updates what you're going to pay throughout the month. Telling you ahead of time what the price will be per month of something would be nice.

> * He was refunded the money AND given credits to make up for the hassle.

Yet, they stubbornly refuse the requests for a hard cap. People were asking for it for years. Some are using ugly hacks like trying prepaid cards: https://www.reddit.com/r/aws/comments/8eaad5/use_a_prepaid_c...

Point #2 is wrong. It would have been absolutely clear had he read the info box or cared. Look at the first screenshot.

Does Aurora Serverless meet your desired criteria, or are you looking for something else?

https://aws.amazon.com/rds/aurora/serverless/

Not quite. Aurora serverless spins up machines for requests and spins them down. If I were paying just for storage and IOs, it'd be perfect. But I'm paying for ACU-hours. There is a 10 minute minimum charge, and an unknown ramp-up time if there are no ACUs spun up.

If I have a web app which is accessed once every 10 minutes (0.002 requests per second), I'll be paying for a full AWS machine. It should be a shared, scalable resource and abstraction.

Reply on news.ycombinator.com