Our support team has reached out to the user from the thread to let them know they're not getting charged for this.
It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
Apologies that this didn't come through in the initial support reply.
“0$ to get started, then pay as you go” reads to me: “0$ to get started, and then you can order add-ons and extra features as you need them”, not “$0 to get started, but we may start charging you virtually unlimited amounts at any point without prior notice”.
When signing up for the “Starter” tier initially, I completely misunderstood this. I didn’t have to enter any credit card or invoice details, so I thought as long as you don’t have that info from me, you can’t and won’t bill anything.
I think I disagree with this, but maybe I'm misunderstanding you.
Pay as you go sounds strongly to me that you pay based on your actual usage, not that it's free except for add-ons. A pay as you go phone, for example, does not imply you need to buy a telephony add-on, an SMS add-on, etc.
PAYG phones, however, were always prepaid, so I think I would expect PAYG hosting to be similar. That said, if my site was publicly accessible without my prepayment, I think it would be clear that it works the way it apparently does.
It's potentially misleading, but I don't think it's intentionally dishonest.
That’s my interpretation as well.
The usage of the term “add-on” is not clear here in my opinion. On their main pricing page[1], Netlify currently lists “Additional bandwidth” as “Add-on”. To me, that sounds like “I can actively order additional bandwidth in case the included bandwidth isn’t enough.” Not: “Additional bandwidth is automatically allocated and charged for as it happens to occur.”
In addition to that, there is a big bold “$0” at the top of the “Starter” plan.
The disagreement is on what "usage" means. I wouldn't assume that "usage" includes things that don't take any action on my part.
If I don't use my phone, for example, I wouldn't get any "usage". A phone pay-as-you-go plan would probably trigger similar outrage if they charged you potentially unlimited amounts for phone calls that hit your voicemail overnight.
The "do you know how it works" is completely unnecessary and rude.
It's really the trade-off for using any cloud host. You are implicitly trusting the host, their monitoring tools, their billing system, and their customer support when things go wrong
Netlify still looks terrible here.
1. Would Netlify forgive the bill if this didn't go viral?
2. How do you plan to address this issue so that it never happens again?
Everyone here knew someone from Netlify would come and say OP wouldn't have to pay. That was a given. Now we want to know the important answers.
2. While I've always favored erring towards keeping people's sites up we are currently working on changing the default behavior to never let free sites incur overages
This isn't what you said in your first post, you said:
> It's currently our policy to not shut down free sites during traffic spikes that doesn't match attack patterns, but instead forgiving any bills from legitimate mistakes after the fact.
So forgiving "lots and lots" doesn't move the needle. Do you or do you not forgive _all_ such cases where your DDOS protection doesn't take down the site? What was your employee referring to when saying that the usual discount is 20%? Are you saying that you _never_ discount 20% and instead always discount 100% i.e. "forgive"?
Solving this only for the free site use case doesn't address the core problem that people are bringing up about a lack of spending limit controls.
i.e. if I know I average $10 a day, I should be able to put in a "If it hits $50, email me and take it offline".
Of course the opposite problem is then people setting that limit too low but since the user defines the limit that's on them not you.
This is one of the reasons I still in 2024 rent physical boxes and run the modern stuff on top of them directly, yes it costs me more per month but the price is hard capped.
What Netlify is doing here is really the best approach for both parties. And typically speaking a $104k bill would be hard to get paid up regardless if the customer's typical transaction balance was $5/mo and their credit card limit wouldn't be that high.
Also, that's the benefits of credit cards - that you can still issue a charge back, and credit card companies very much favor the consumer rather than the merchant.
There would be many attorneys interested in collecting a $105k debt.
I mean I'm not familiar with every debt collection scenario under the sun but Internet randos seem to think this is a real thing where like a cloud/hosting company sends an army of lawyers to repo some guy's house and runs him into bankruptcy because of a traffic overage. I've never seen it work that way, what happens like with most business debts, is someone at the company negotiates with the debtor to try and get as much out of them as they can, and failing that, possibly refers it to a collections agency which does the same but plays a bit more hardball.
In the case here with Netlify even before it went viral they reduced the amount from $104K to $5K, no lawyers, collectors or repo men involved, and while I'd hate to be stuck with that $5K bill, I dunno, that does feel closer to the mark of something that maybe you should be on the hook for if you're responsible for 200 TB of bandwidth overage over 4 days? Is this so bad on the part of Netlify?
All that said I'll just add that I've never given my credit card to any sort of host/cloud who had terms where they could bill unlimited overage fees like this. Never will unless there's a cap. Not Netlify not AWS not nobody. That goes for my personal life as well as for the business I operate. The terms is the terms and the answer is to not use these services unless you can afford them imho.
The responsibility part is the tricky part of the equation.
If someone hits your site with a DDoS attack, are you responsible? There's literally nothing[0] you can do as a customer of a cloud provider here because anything you can do is limited to the servers and services you're given access to. For example even if I had access to billions of requests and built an anti-DDoS tool it would still need to run within the cloud provider's provisioned server which means I'd be on the hook for all traffic costs because it's something running in my account.
That doesn't seem reasonable to me as a customer. It means a cloud hosting provider can put an extreme financial burden on a customer and make a killing in profits because of the markup they charge on bandwidth. The incentives are terribly misaligned.
[0]: I mean you can sign up for DDoS protection through a 3rd party company but in this case I'm talking about taking actions within your hosting provider.
From personal experience as a customer of a cloud provider (not with Netlify btw), usually cloud providers who profit from bandwidth costs will write their TOS in such a way where almost nothing qualifies as a DDoS attack unless it's truly a distributed and targeted large scale attack specifically on your site.
A random person on the internet who spins up a few VPSs around the world and slams your site with looped curl requests won't count as a DDoS attack even though from your perspective that will result in a massive bill increase due to bandwidth costs.
In other words, I'm not surprised "didn't match attack patterns" was used. I'm guessing that will be the case most of the time.
Mostly false. Either transit is billed on a 95th%ile basis (so...more money for more traffic), or if it is flat/netted, you're still paying for the capex for the switch ports (fatter connection to support more traffic means more $$$ for the gear to support it).
You can say "the sum total of free-tier traffic requires X additional connections" but the sudden burst of DoS traffic did not raise marginal costs (besides an inconsequential usage of electricity).
So your suggestion is to issue a chargeback.. to get money back that should under the terms of whatever service you signed up for be owed?.
That seems like bordering on fraud tbh.
> Additionally, when you set that limit people then get upset because usually when they go over it for a good reason, like going viral, they aren't anticipating it, and just when their traffic is most valuable the site is down.
Legit concern and something I mentioned, I'm gonna guess there are broad two camps on that one - mine which is "I want a safety ripcord" and "whee, nice problem to have".
However since this entire conversation is around a guy who got a massive invoice because of a bill he wasn't expecting and couldn't have set such a limit I'm still gonna go with a "I want a way to constrain the financial downside - hell turn it off by default but give me the option".
Since broadly a lot of cloud stuff doesn't, I'll constrain it a different way.
Funny story... One of the big cloud providers actually has you do that on purpose as a remedy for an account you've lost access to.
If they're about to go over, shoot them a quick heads-up and give them 24 hours to sort things out or level up their package.
After that, if they haven't made any changes, temporarily pause their site access
By the way, your comment was flagged which seemed odd, so I looked at your profile and it seems like all of your comments are being (automatically?) flagged and don't appear on HN by default. You might want to talk to the HN staff about that.
2. To me, it's more likely they don't have one because there are edge cases to consider that make "hard limits" difficult to implement. What is AWS supposed to do when you hit the limit? Is it a hard limit? Ok, so when I hit my budget, all my s3 buckets get deleted and all my EBS drives get dropped? Do all my code deployments just get deleted? Do you "bless" certain services so that they continue to charge the user even after the hard limit? How is all of this communicated?
You can set an alarm in AWS today and the user can decide what to shut off. If you really need to, you can create a script that can hard nuke your account once a limit is reached; but I don't see why AWS should nuke your account for you.
If providing this bandwidth cost them $50k rather than $5, I would bet you my entire life savings that they would QUICKLY find a way to add hard limits to their service, no matter what technical challenges they're quoting now.
2. This is probably a 95/5 problem. The vast majority of these sorts of cases that I've seen and read about are caused by increased traffic, which hits the customer either with extortionate egress fees or unintended compute scaling behavior (FaaS/VMs).
Storage is trickier, but you could stop accepting writes or reads after passing some hard limit.
This leaves some edge cases, sure, but it should handle the vast majority of unexpected bills without any destructive actions.
> You can set an alarm in AWS today and the user can decide what to shut off.
That shifts responsibility back on the customer, which is exactly the problem to begin with. I need assurances that I won't be billed more than $X in any given day, or month and this solution doesn't provide that. Maybe their API down, maybe it's serving stale data, or maybe my automation fails for some reason.
> How is all of this communicated?
Hard monthly limits:
Egress: [ $ 10 ] - When exceeded, all outbound traffic is limited to [ 0 Mbps ].
Compute: [ $ 10 ] - When exceeded, scale all compute instances to [ 0 ]
Data Storage: [ 1 TB ] - When exceeded, all writes are rejected
Reads Ops: [ $ 1 ] - When exceeded, all reads ops are rejected
It's really not that hard. Offer these limits on a per-project, or even per-resource level, that would naturally allow you to limit the blast radius. A personal website that has gone viral would likely want to have different limits than an email server under the same account, for example.
No, I believe you are assuming that proving a "hard limit" feature is cheaper than just refunding people from time to time. If you consider the all the products AWS has to offer, all the different ways they are billed, then figuring out what do to for each product once that limit is reached, and potentially doing a destructive action, and then all the code and testing on top of that - it seems far easier to add a human in the loop to just refund people on a case by case basis. It's a ton of complexity for likely a rare issue, and if someone really needs it they can build one themselves and choose how to handle what needs to be done once the limit is reached.
Building all this out just so that some college kid isn't charged a bill they could obviously never pay likely isn't a good use of resources. It's not likely AWS has a reputation of being greedy either, if you explain the situation they refund you. If they are truly doing this to make money they have remarkably poor execution.
>It's really not that hard.
Storage pricing isn't done up front. You pay per gigabyte-month. If your "hard limit" kicks in, then what does Amazon do with the data you are no longer paying to store? Does Amazon drop your entire database once the credit limit is reached? There are plenty of Amazon services like this. Consider secret manager. You are charged 40 cents/month per secret. You have 100 secrets, so $40/mo, but you set your hard limit to $20. The middle of the month rolls around, what does Amazon do? Do they just drop all your encryption keys?
There are 100s of AWS services and many where you can't just apply a sensible rejection policy once you hit that hard limit without doing something the user cannot recover from. It's only not hard because you aren't actually thinking about the matrix of AWS products that exist and how they are billed.
I don't think you understand what a hard limit is. When I set a hard limit, that means that under no circumstance should I be billed more than this amount. From this you should be able to deduce that the platform needs to stop you from ever entering into a situation where they could end up having to decide between nuking your data, or overcharging you.
This means that for all data storage scenarios, they should apply limits to projected monthly usage rather than actual monthly usage.
If 1 GB of storage costs $1 per month, and my monthly limit is set to $10, then they should not allow me store more than 10GB of data, rejecting writes as needed. This design guarantees that my hard limit won't be exceeded and the data is never at risk.
Any service which doesn't persist data can apply limits based on actual usage, and safely shut them down when the limit is exceeded.
> Does Amazon drop your entire database once the credit limit is reached?
No, they should refuse further writes when your database reaches a size that would exceed $10 a month. That's a safe behavior.
> Consider secret manager. You are charged 40 cents/month per secret. You have 100 secrets, so $40/mo, but you set your hard limit to $20.
If you set your hard limit to $20, then Amazon should refuse to create more than 50 secrets. That's a safe behavior which limits your spend to at most $20.
> Building all this out just so that some college kid isn't charged a bill they could obviously never pay likely isn't a good use of resources. It's not likely AWS has a reputation of being greedy either, if you explain the situation they refund you. If they are truly doing this to make money they have remarkably poor execution.
This is just a bad faith argument, everyone from broke college students to medium sized businesses has complained about this at one point or another. Relying on the good will of a megacorporation to forgive a surprise $100k bill is nuts and if you scroll through these comments you'll find some that claim that AWS is no longer forgiving these bills like they used to.
I'm not claiming that this solution is perfect, or that it would cover every failure mode, but it sure beats the status quo and would solve the vast majority of surprise bills that result from unexpected traffic.
>No, they should refuse further writes when your database reaches a size that would exceed $10 a month. That's a safe behavior.
I think we have gotten to the point where we've lost the plot here. You previously stated "It's not that hard". We are doing projection forecasting! And the storage layer for these range of services (S3, MySQL, Postgres, MSSQL, Kafka, Keyspaces, DynamoDB, probably 20 more im missing.) must now either call out to some financial projections service and reject writes for billing reasons. On top of that if you have a hard limit you are locked out of dynamic scaling. Need to 100 nodes for an hour? Nope, you now must disable billing limits because Amazon assumes you will use every resource for the entire month. Surely someone wont disable billing limits for an hour and forget to re-enable them, right?
>but it sure beats the status quo
I'm not sure handicapping every service for some barely functional billing limit beats the status quo. It's certainly more expensive for amazon in terms of engineering load but with so many footguns I don't see how it ends up anyway other than Amazon having this feature and still having to do service refunds because billing limits were disabled due to awkward limitations.
>This is just a bad faith argument, everyone from broke college students to medium sized businesses has complained about this at one point or another. Relying on the good will of a megacorporation to forgive a surprise $100k bill is nuts and if you scroll through these comments you'll find some that claim that AWS is no longer forgiving these bills like they used to.
Medium sized businesses have had their bills forgiven for things like cryptomining for example. My point is billing limits don't really work for AWS, alerts is the best you will get because AWS doing destructive actions for you or assuming your usage patterns is worse. It's not an easy feature and to imply they keep the status quo solely because Bezos needs another yacht is reductive of the problem.
It's not a complex "forecast". It's simply the monthly rate as I've illustrated with numerous examples.
If my limit is $10, then don't let me occupy more storage than that. It's a 1:1 conversation between gigabytes and how much they cost on a monthly basis.
Do you have ties to cloud providers you'd like to disclose? It's a very simple concept that you're refusing to understand, which usually happens when your job depends on it.
I have 100 secrets and no hard limit. Now I set the hard limit to $20. What does AWS do?
Additionally, when you set that limit people then get upset because usually when they go over it for a good reason, like going viral, they aren't anticipating it, and just when their traffic is most valuable the site is down.
The fact that most cloud operators don't have actual hard cutoffs to maintain financial responsibility is intentional. Azure does, but only for specific account types. If it's PAYG, you can't do it. The end result is if you do something "weird", or someone DDoS's you, you're liable.
With a hard limit, a DDoS just takes your site offline.
I agree. I also agree that when dealing with large numbers of people, there will be people who don't understand this and/or will actively try to social engineer their way out of their own decisions.
Setting customer expectations and meeting them successfully isn't easy.
Instead of forcing user to set a low cost limit and missing a viral opportunity or the platform writing off the massive bill the customer can't afford... just put the billing mode into a reduced price mode or have some more nuanced configurations. Sometimes is just asking the question the right way. Instead of "max spend limit" or similar "If your site goes viral, how many requests do you want to serve before going offline? 1M=$20, 10M=$100, etc" at this point, I feel like bandwidth consumption is a bad metric for billing; just use requests/visits/actions and price for those.
This is not prescriptive just illustrative. The point is make a better pricing formula to account for these massively unexpected events. Couple it with an aggressive notification policy when this traffic event gets triggered. The user should know the traffic pattern has changed and a high traffic event is happening. They can login and change the configs and decide if they want to keep it going or not.
Most users don't want to be banned from their hosting provider.
Of course. And that’s why any limit against a dynamic variable should also have alerts linked to it
Send an alert to the user when traffic starts spiking, especially if a simple projection shows it’s going to go over their limit
Then the user is aware, hopefully with enough time to lift the limit if needed
If your customer is aware enough to notice they are being hit with a DOS or legit traffic while it is happening, then great! They can respond, and if needed, engage proserve to get support for scaling or defense depending on needs.
If your customer is not alert enough, then their site is offline, and they won't hear about it until their customers are screaming at them, which will result in a P1 ticket to look at a vendor who won't turn them off during an unexpected peak.
It's a catch 22, and if you have to choose between: a) a PR hit because you have to go on a forum and post about waiving the fee, or b) a PR hit because someone posted a blog post about how you killed their site during a moment of critical growth
any reasonable business will choose A every time because A is far more supportive of customer growth and has drastically better optics. Anyone who thinks A is worse is probably too inexperienced to have an opinion.
I'd say the same about those who only think in absolutes.
That's a false dilemma. You can give your customers a choice in these matters with an optional hard limit, which I realize seems like a rather extreme idea these days.
Someone running a personal project will likely opt to have a low hard limit, say $10, while businesses will more likely opt in for alerts without, or with very high hard limits.
Hard limit: [$1000]/mth or [$500]/24hr period
Notify me if traffic exceeds thresholds: [$800+]/mth or [$400]/24 hr period
Notify me if the traffic forecast for the month looks like it will exceed my hard limit before the end of the month.
That has not been my experience. I've had to do a few chargebacks for services not rendered, and I've never won. I will submit my evidence, then the vendor will submit 100 pages of random emails, and then I will have my claim denied. Then I will appeal, will point out that they sent 100 random pages of email, and then they will reply with the same 100 pages of emails and I'll get denied again.
It seems that the vendors have found the hack for chargebacks -- just inundate the credit card company with so much data that they assume the vendor must be right.
It makes sense -- the vendors pay the credit card companies a lot more than I do. They'd rather keep them happy than me.
Plenty of morons just use the service and chargeback right before renewal so they got the service for free, some just chargeback instead of cancelling their plan or asking a refund. I get hit with 15$ bill plus I lose all the money even if I provided the service.
Whatever email / data from my system I send is ignored and the scammer / moron gets his money back.
I'm sure that's how it works if the vendor is large enough.
If you are a small fish you don't stand a chance, which is why for my next project (where I'm supposed to charge a lot and spend a lot on behalf of the user - and I'll be royally screwed if I start getting chargebacks on 500$ of which I've already spent 450$) I'll just accept crypto payments.
1) Thank you for allowing us to set the limit.
2) I understand your opinion that you prefer chargebacks but I disagree with it.
The very reason I stay with Hetzner is that I know in advance what my bills will be for the whole year. Heck, I even charge my account in advance so that I don't worry about any charges!
Like let them go over one month and then make them sign up for a paid plan for the next.
Sort of, but the approach to the approach isn't great. If you're going to void charges from DDOS traffic anyway, you might as well make that explicit policy, rather than doing it after the fact in a way that seems discretionary.
The Reddit thread is full of people who are saying that they intend to pull their static content off of Netlify and move it to Cloudflare Pages, which has no overages on the free tier in the first place. I can tell you that I personally chose Cloudflare Pages to set up a new static site a couple of weeks ago, and Netlify wasn't even in the running.
If Netlify's free tier is important to their customer acquisition strategy, then they really need to retool how they're offering it, or Cloudflare is going to eat their lunch.
Also: what's common? At the scale these companies so desire, small percentages are still thousands of users. Supporting them with what's ultimately a fairly trivial option shouldn't be seen as such a hassle.
I still prefer this too. Kinda funny how server resource limitations became a feature and not a bug when it was one of the problems the cloud sought to overcome
It's bizarre to me that more services don't support billing this way, since there are tons of situations where I would much rather have a site or service go down than be hit with a surprise bill and have to depend on social media and magnanimous corporate PR.
You can do stuff like "disallow any of these instances to be used in your env", so if you never use graphics cards, disallow the whole class.
You can also set limits like "no more than 20x m5.4xlarge".
But again, AWS is the worst about no actual hard limits, cause each system generates bills. Ive also seen the hell of "hidden system AWS Billing doesnt have is still submitting billing and we dont know what it is". Again, AWS enables basically infinite liability.
Ive also discussed with C levels that "every engineer and dev with AWS logins have an unlimited credit card to of which you're on the hook for". Lets just say that 'heartburn' doesnt even begin to describe the terror on their faces.
The big advantage to "cloud" was that you can provision more resources in seconds. Existing data centers had terrible non-VM and non-good VM management software that provisioning more CPU, RAM, storage was a weeks or months long trial.
And you can still get a 100k bill for hosting a SINGLE text file, cause DDoS and shitty hosting providers who demand unlimited credit is a thing.
That said, I still think there should be an easy way to set a daily limit. Should I use the Budget service to do that? Cost Explorer? Billing Alarms? Is it possible to have them shed whatever's spending all the money?...
Again, I see the whole can of worms here: what if your service is jamming tons of data into S3 because of a bug? Or you actually started something that got popular and you have a gigantic Dynamo table? Stopping an EC2 instance is maybe an easy call, but deleting data is iffy.
AWS just feels like a minefield because I'm occasionally worried with all the products, I'll check a box when creating an instance or SG or whatever, and that'll (e.g.) trigger CloudWatch to read all my logs, but I have some crappy debug config for some app which will vomit out dozens of logs a second accidentally, and instead of just trashing `/var/log/` I get billed for millions of log events or something.
They might even offer this as an insurance, so you pay a little more but can be sure you stay in budget.
How much is unlimited talk, text, data, hotspot with a watch data add-on where you are from?
edit: Actually I was wrong–it is $30, not $35.
I pay $20 for the phone plan, and $10 for the watch add-on. No fees or taxes Feels reasonable to me.
I could also get 50 MB data, 2h calls, unlimited texts for 2€.
I stopped ~all autopays when (Boost Mobile?) deduced 200 instead of 20usd from my debit account one month in college. They refunded the difference relatively quickly, but I racked up 5 or 6 overcharge fees before realizing; ended up being a pita for an already broke 18 y/o to figure out.
I was a broke and stupid kid. Never use a debit as an autopay. But since then I like to track where each penny goes as it goes.
But this is a good reminder why my gut feeling always made me avoid these overengineered solutions.
They do. But of course maximizing profit is the sole true prerogative of capitalist enterprises. And the market is not totally competitive. So yes your intuition was correct, to be cautious against over enginnered pricing to get y'a.
I mean those companies cater to hobbyist. Then ...
Render seems more fair-play. Until a change of mgt occurs of course.
>We normally discount these kinds of attacks to about 20% of the cost, which would make your new bill $20,900. I've currently reduced it to about 5%, which is $5,225.
20% and 5% are quite a bit higher than forgiven.
I wouldn't want to be CEO these days. A lot are trained and paid to do damage control.
With all that money they then can finance the free tier until they get too far and become platform locked-in.
Surge.sh Im not sure. But shows all the sign of some greedy acquisition, regular long outages , as if I have been sitting as a free tier for too long, quick nudge to pay. For barely accessed sites even behind CDNs, steep. I even worry they one day just wipe all my buckets (they did for a few already) and support would recommend me to be a "normal" paying user .
Nothing is free. And nothing too good to be true is true .
No offence, but this sounds like "trust me bro" billing and it is not good enough. Someone could literally get a heart attack from getting $100,000 bill - this amount of debt can literally ruin someone financially.
> 2. While I've always favored erring towards keeping people's sites up we are currently working on changing the default behavior to never let free sites incur overages
I hope you understand that chance someone who used to pay you $20 / month unlikely want to ever get $10,000 bill. Yeah people might dislike that their website went down due high traffic, but it's not gonna bring this much negative PR as incidents like this. There should be some sanity check at least.
Not a heart attack but there has literally been at least one suicide over an unexpected life-ruining bill like that:
https://www.forbes.com/sites/sergeiklebnikov/2020/06/17/20-y...
Hey I'm a taxi driver. Hailer fell asleep on the back, so I kept driving all night, once he woke up I dropped him to his place and asked for my monthly wage. I "forgive" many, but just a few are juicy income so I adopted the policy to never wake any customer up. If people ask I say it would be impolite, principles prime.
That’s the elephant in the room here. I understand an enterprise plan where you state billing is $xx per GB, but billing someone with a free site??
Give me a break.
Sequence of events doesn't support this answer:
1. User gets charged 100k
2. User complains to support
3. User receives discount to 20k, then 5k. Support states policy is normally 20k
4. User discloses to the world. Goes viral.
5. Invoice is forgiven
While you might forgive "lots and lots", fact is that you still presented the invoice to a free tier customer, and when they complained you gave them a discount, but still charge. Only when it went viral did you forgive it.
I'm a paranoid person by nature so "It's free... just... give us your card details" is always suspicious.
1. What are you doing to prevent DDoS's from hitting your network?
2. Why do customers have to allow an unlimited credit burden to use services?
3. Why arent there cost controls to "if $$ exceeds X, shut acct down"? Azure can do this.
Long story short, why are you by default (except for social media escalation) passing fraud costs to customers?
From what I see you could change this policy tomorrow unilaterally and we would have no recourse.
I wouldn't think it's a binding policy at all, because the billing procedure (automatic full bill, manually discounted bill, etc.) would follow it if it were. More of a procedure.
Honestly, this terrifies me---I run a bunch of different sites off netlify, and I would have never imagined that a site could jump from 0 to six figures of bills a month without something hitting a tripwire somewhere and cutting it off or at least communicating with the account owner. At least users should have the capacity to self-impose bandwidth caps to prevent this sort of thing.
It seems that significant bills appearing without warning or cut-offs is clearly intentional. I am embarrassed that I recommended Netlify before.
To bobfunk, the response needs more empathy and explanation around the obvious frustration around why there is no slider for cost limitation.
As it is, it feels like the minimum viable corpspeak apology and damage control.
With models like this and AWS people will get afraid of success
The average HNer seems to be recommending colocating your physical server :-)
Before the internet, these issues would be handled by local news journalism, and still sometimes do!
the fact that once it arrives to the limits does not display an error page.
At this point I honestly do not care about they changing their policy, they should have thought that a normal person receiving a 100000$ bill on a free tier shall not been at all on the table in any circumstance, even if they forgive the bill, nobody needs to stress out like that.
The only "fix" here is to act like Hetzner and null route upon DDoS, price cap the thing, or offer unlimited bandwidth on the free tier like e.g. Cloudflare Pages.
Uncapped but paid is a recipe for disaster and you'll always be subject to the will of the support staff when something happens. If they can grasp to a straw leading to suspicions that it's not in fact a DDoS attack, you can for example be sure they'll do just that. Just no.
https://www.hetzner.com/dedicated-rootserver/matrix-ax/
With a 48 core Epyc or 80 core arm server, one really shouldnt need much more for a middling project. There are enterprises who run entire services on such hardware.
If there was an option to cap billing, or at least some legally binding limit on liability, then I can countenance using netlify.
Until then, it's just not feasible nor worth the risk.
If your web site makes you money in proportion to the amount of views or bandwidth you use, by all means, go with a provider that increases your costs when your traffic rises. But if your web site does not make you money, why not host it somewhere for a flat rate?
I've been on a $25 instance for years for all my sites, which worked as well.
My only hesitation is the 20k file limit; one of our sites is a large Gatsby site that generates certain content programmatically, sometimes using a series of YAML files as the source. Pretty sure we’re over 20k pages by themselves. Been meaning to move this one to Next.js sometime but that’s not on our roadmap anytime soon.
I'm not sure about VueJS specifically, but I run everything I can off a $6/m digital ocean droplet (static sites, web apps, git repos, RDBMS, some other custom apps I've written) and it hasn't broken a sweat yet[1].
My understanding used to be that requests will be dropped if my virtual server can't handle it, and I'll have to transfer 10,000TB to get to a $100,000 bill.
In practice, my server will not physically handle the load to serve more than maybe $1000 of data a month; it will fall over before that.
In summary, using a VPS is sorta like an instant hard cap.
[1] Until I tried using Jenkins. Which crashed constantly because apparently 512GB of RAM is too little for what it does. I'm now in the process of writing my own little CD tool that isn't going to go over 30MB of RAM just to run my deployment scripts.
This is the big thing, but I also think that modern out-the-box OSes are pretty damn secure these days.
IOW, the amount of knowledge and time needed to maintain my single VPS is a lot less than the knowledge and time you will need to manage your costs using multiple hosted SaaS suppliers for static hosting, web-app hosting, database hosting, repo hosting, etc.
I did the same last night from my phone. My personal site and a project docs site are just going to not be online for a couple days. Easy choice.
This is a serious matter. We are building a new site for our company with Netlify, but we can't open ourselves to this predatory practice. And even if you do not mean to be predatory, even the option of such is enough.
If not resolved with a clean, legally binding promise, our company (and probably quite a few others) must move our business to Cloudflare, Amazon, or some other competitor of yours.
Is that unreasonable?
edit: hey guess what, Netlify offers an enterprise plan, I'd bet they will be happy to offer you a "clean, legally binding promise": https://www.netlify.com/pricing/?category=enterprise
"Didn't come through" doesn't actually match the user's report of having support explicitly offering 20% and then 5% payment. It sounds like maybe you have a training problem? That seems like one of the important points to speak to.
That doesn't square with the 5% fee on the original $104k that your company told the OP to then pay.
Well, giving the option to users to plan ahead would be best, no? Like a setting to choose whether they want a potentially unlimited bill versus downtime. Instead of that, you are choosing to stress and make people scared/anxious/homeless even (if they don't think of raising the issue on HN).
Seriously, this is not rocket science. This must have been discussed before in your company, and someone actually made this decision to stress people about such bills.
The legitimate mistake sounds to be on _your_ side if anything. You failed to match the attack pattern after all.
> Apologies that this didn't come through in the initial support reply.
The support email said you normally discount the attacks to 20%, but in this case it would be discounted to 5%. Are you here publicly claiming that your policy is to in fact to forgive (i.e. discount 100%) these bills? Was the support reply totally incorrect in claiming that you normally discount the attacks to 20% or are you lying when saying that your policy is to forgive the bills? You might want to clarify your position here.
This is a static site. To reach that sort of bandwidth out of nowhere you'd need to publish the blueprint for a teleportation machine
That said, instead of depending on unreliable heuristics, they should just allow an option to change the behavior. The "current policy" to charge small sites on the free tier thousands of dollars instead of just throttling/shutting down the traffic is really predatory.
1 million downloads (= visits) is nothing for "going viral overnight"
I wager the vast majority of people in the free tier would gladly cap their traffic at the (generous!) bandwidth offered by Netlify. Even to the majority in paid tiers, 100k bills where there previously was none must be unwanted and unintended.
I mean, we all know dark patterns are a thing...
You can avoid this sort of bad press and disgruntled users and your support cost by just giving users the option to shut down the site once the bandwidth budget is up.
Moving my sites off of netlify ASAP.
About this specific case: it set off the flamewar detector (a.k.a. the overheated discussion detector) and also got moderation downweights. We sometimes turn off that penalty, but I don't think we'd do so in a case like this, because HN gets so many posts of this nature. They flare up with Big Drama that is sensational for a while but not particularly interesting, and therefore not really what the site is for.
In fact HN gets so many posts of this type that it has become a joke, and not only that but a cliché, so much so that the top comment of the Reddit thread repeats it [1]. That's about as repetitive as anything gets. The basic idea of HN is to gratify intellectual curiosity [2] and avoid repetition [3].
[1] https://old.reddit.com/r/webdev/comments/1b14bty/netlify_jus...
[2] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
[3] https://hn.algolia.com/?dateRange=all&page=0&prefix=false&so...
I read this whole thread before the CEO posted and after, and neither time thought any of the comments were out of line or even that the general mood was any more heated than any other random HN thread. People are politely asking pertinent questions.
And I think once the CEO makes a statement which contradicts the company's support response, that becomes very interesting. Particularly to anybody that uses their service. I'm certainly not finding the conversation very repetetive or cliche.
I can tell you pretty much for certain though, that we'd hear many more complaints if a Reddit thread about a customer support shitstorm stayed on HN's front page for very long.
Btw, the Customer Support Fuckup category is one of several $X where HN has become known as the place for $X, but only because HN is not actually for $X. Another example is the Site Is Down category—people often come to HN to find out what's going on when some $Site or other is having an outage. But just as HN itself isn't a site monitoring platform, it's also not a customer-support-of-last-resort platform.
If the community feels like this customer support fuckup is altogether more interesting, I'd consider reversing the call, but again, my gut feeling is that we'd get even more complaints that way.
I learnt more about Netlify, Vercel etc and how they operate from this thread from the last 100 "customer service" threads combined. I learned about Cloudflare's offerings, and a bit about Hetzner. And it was all very interesting.
You said you sometimes turn off those penalties, I think this thread would be a good candidate.
"Let's pretend it isn't," said Rabbit, "and let's see what happens."
[1] https://tinyapps.org/blog/202402260700_netlify-to-cloudflare...
Something that makes me feel uneasy about the fact that the post gets hidden is that this strongly benefits Netlify. It seemed like lots of people moved off Netlify after reading the post.
I'm not suggesting that HN actively took an action in Netlify's favor, but the potential is there. Is the algorithm for flame war detection open source? Or do we essentially need to trust you that there was no interference from Netlify? (I do trust you but others might not).
I don't know how to get every user to trust us. All we can do is answer questions when asked. That seems to be enough to satisfy most of the community most of the time, and it's probably not possible to do a lot better than that, much as I would like to.
OP was ignorant, and got tossed a lifeline. Also “just make everything zero dollars bro” is a ridiculous proposition.
I have no control over how much traffic my public sites get. There is zero value in me signing up for a service which charges me based on traffic if I can’t control the maximum they’ll charge me. Would you sign up for an infinite bill?
This is a dumb analogy, but the point is there is very obviously a pattern in this payment process that is ripe for abuse. The question of whether or not you aim to be an abusive business, plucking every shady profit where you can put the onus on the customer to try to come get their money back is one that many companies are deciding, and many are erring in the direction of the dark pattern.
By not working to avoid this problem from the get go, there is an implication about how a company wants to make their profits.
If it has always been a "policy" to forgive bills, shouldn't it have been 100% forgiven immediately after OP contacted support in the first place? Why go through the trouble of playing the hero by offering "discounts".
After this the CEO comes along and says that the policy is actually not to bill for this kind of event... But the company actually tried to bill this user 3 times... soo it all stinks really.
Do you forgive 100%, 95%, or 80% of the bill?
Is the 100% only available when a story about a bill goes viral?
This large bill doesn’t look like a legitimate mistake, it looks like everything worked as intended until things got escalated via Hacker News.
Not only should this stuff be capped rather than the dam allowed to flow, but your systems should have picked this up immediately and known it for its nature.
Thus must have been a nice little earner for you over the years.
I'm moving all my netlify sites elsewhere, bob.
I'm probably not the only one.
> "You've got room to grow!"
ohfuckohfuckohfuck
I interpret this as "we always charge for traffic served, but we attempt to block illegitimate traffic" which means of course that the worse their traffic discriminator performs, the more money they make!
One question though, what is Netlify gonna do to ensure this doesn't happen again?
I understand it's a hairy question, but the general consensus seems to be some policy must be changed or at least some line should be drawn.
but instead forgiving any bills from legitimate mistakes after the fact
That's terrible for marketing.
It may only move a few MB a month, but I just can't risk if I put anything more substantial there that I might get hit with a bill for $100k and you maybe will forgive it. And that this has apparently been policy for nearly a decade makes it even worse.
Sure, this instance was resolved, but it's also the top post of the last month. Who honestly things it would be the same outcome if not for going viral...
I wont touch a fake free service if it requires a payment method. Want my money, give me a reason to pay you, dont trick me into paying you.
Temped to go fuzz your product and document other dark patterns...
Collect this from my corpse you scummy fucks.
Truly shameful.
Your support was going to charge him 5% as a "sign of good fate". How kind.
If it hadn't gotten traction, you absolutely would have charged him.
How many other people have you strong armed into paying ridiculous bills?
The fact that you have no usage limits is clear indication that this is intentionally left open to abuse.
Extremely shady and downright criminal.