Hacker News Reader

Comment by jzebedee | Hacker News Reader

The most bizarre thing is that this is a known issue that folks have asked them for ways to mitigate, to no avail. The reddit thread even links to an extremely weird dialogue where Netlify's response boils down to, "if you're hosting a small site that gets DDoS'd, don't."

https://answers.netlify.com/t/limit-bandwidth-to-avoid-high-...

View on HN

https://www.netlify.com/security/ sez “Active DDoS mitigation — Netlify monitors for traffic pattern anomalies and spikes, and effectively controls for them as needed” and now I'm curious about what that actually means.

It means they protect themselves from layer 3 and 4 DDoS. For layer 7 you're mostly on your own. That's what most companies mean when they talk about DDoS anyway.

I found https://www.netlify.com/blog/2017/03/28/why-you-dont-need-cl... and it sounds like you're right.

“The cool thing is that we also provide a load balancer, and if our system has detected that our main load balancer is currently being hit by a large DDoS attack and is slow or unresponsive, we’ll simply route around that on the DNS level. Since we cache content at our edge nodes around the world, end users also experience extremely fast page load times because of this.”

Well at least Cloudflare will not charge me for this situation.

Right and as a CDN they HAVE to handle layer 3 & 4 DDoS themselves so it's not like they're doing you any favours. The traffic is typically routed to the customer based on SNI.

It means that they will charge you 20k (a year's rent for me, no biggie) instead of 100k for your free website, or 5k if you got lucky.

If you value uptime, even through being massively attacked, they can offer you that.

That is not unique to this price point and most of their competitors do not charge for unusual traffic spikes.

It's a static website. Plenty can offer uptime through being massively attacked, for free.

OT: It seems like your comments are being automatically flagged/killed by HN, you might want to reach out to HN staff. You're the second person in this situation I've come across in the past hour, odd.

I had the intuition that Netlify are extremely incompetent compared to Cloudflare, and this thread adds another data point. So no, if you value uptime you are not going to rely on them.

Who on earth does this describe?

1. Comfortable paying >$100,000 to prevent their site going down for a single day, and

2. Doesn't pay a dime for their hosting service on a day-to-day basis

They reroute the network traffic to ensure none of it gets dropped so they can accurately overcharge you for the the correct amount.

I'm hesitant to use "fancy" cloud service/hosting providers for reasons like this.

I don't understand why they won't just raise a 503 if the traffic exceeds the spend limit, or at the very least provide that as an option.

What's spend limit?

Autoscaling is a feature!

I guess we need regulation for this.

Or, rather than creating more regulations, people could read the contracts they agree to when they get service, and use a competitor if they don't like it

Yeah exactly why we need regulation - so contracts aren’t 100 pages long. DRY.

So, go to a different provider?

More like don't start a business at all because of provider risk.

Or use a provider with a predictable cost structure. There are PLENTY. You didn't need to choose this exact one.

The problem is that it isn't entirely clear which ones have predictable cost to the non-lawyers eye. I.e. they should have to have sane defaults, like reasonable spending limits and opt-out, by regulation, since the market is failing here.

It was clear though. Does Netlify not tell you their bandwidth pricing?

I don't know many people knowledgeable enough to read and understand legalese. Except lawyers ofc.

There is such a thing as an unfair contract. Moreover, there are business practices that can become a local maxima in an industry and squeeze out competition which would actually be a net benefit to most consumers. Mobile roaming was the same before the EU intervened, and were now going back to the shitshow tjat preceded it in the uk.

Why do we need regulation? "Keep the service up no matter what happens, no matter the cost" is a useful business model for companies that make the mistake of promising too many nines to their customers.

The issue at hand is that people put small websites on hosting providers designed for megacorporation wealth, like Netlify. I highly doubt the average blog needs more than a $10 VPS located in one single country without automatic fallback to another data centre. You can probably even go with a $5 VPS if you don't care about the first wave of HN front page bots not being able to reach your site.

> "Keep the service up no matter what happens, no matter the cost" is a useful business model

I mean, yeah - but that shouldn't be the default and it shouldn't be something that you can't opt out of if it is, which is what sounds like happened with Netlify.

Why would Netlify offer the option to opt out when extreme availability is their core business? I'd argue that people are using the wrong service provider if they need to opt out in the first place.

Same goes for most of the other pay-as-you-go providers that turn HN into billing support every now and then; very rarely do I see "we suddenly got a $20k bill" posts about services that these extreme availability products make sense for.

> You can probably even go with a $5 VPS...

You wouldn't believe the amount of times I've said this and the response was "but it costs me nothing right now"...

We don't need regulation for everything. Let customers vote with their wallet or even start a few lawsuits.

Adding more regulation makes the system slower.

Regulation is needed for lawsuits

I'd be in favour of a regulation to allow them to set a spend limit, opt-out.

I'm fine with their pricing structure right now, since you have PLENTY of providers to choose from - people can easily vote with their wallets and there's no problem that needs solving.

However, the unexpected spikes are a problem, and providers seemingly don't provide any way to solve them because they make more money by not solving them. A regulation to require all providers of post-billed services to provide spending limits would make a lot of sense.

Of course, customers should also have the option to opt out of the limit or set a very high limit.

This should apply to any service that's billed by usage calculated afterwards, not just web hosting, and not just technology.

Microsoft is the only cloud provider with a spending limit. One choice is not competition. That's why regulation is needed.

Vercel offers spend limits: https://vercel.com/blog/introducing-spend-management-realtim...

Playing ”devil’s” advocate: tracking spend in real-time is not trivial. It adds complexity to stack. Bugs in the feature can cause sites to go down (for long time) without a reason. Larger online businesses likely rather sort out the problems later than risk shutting down in the middle of unexpected success.

(But I also would like to see this feature)

Vercel will happily tell you how much you are spending in pretty much realtime as it sails past your budget

OP is right though, realtime alerting is non-trivial to build. It looks us a lot of work at Vercel to get right. We also offer budgeting options where you can set spend limits now, too.

Oh wow sorry didn't mean to be personally flippant to the VP! Hacker news is wild.

I just had a look at the billing screen and I can see notifications but not budgeting options (on the Pro plan). Ideally I'd like to monitor/configure spend per deployment as some of our sites have big sales which can = big fees. Is this possible?

Not really. AWS has budget alerts right? And I can read those budget alerts through their API.

So it would be trivial for me to poll their budget API for an alert, and immediatly trigger a shutdown of my Cloudfront service. Why can't they do that for me?

"AWS Budgets information is updated up to three times a day. Updates typically occur 8–12 hours after the previous update" [1]

Something based on this could be definitely better than nothing, but might also give false impression of safety.

[1] https://docs.aws.amazon.com/cost-management/latest/userguide...

It's something. I started looking into the budget alert docs and it does use SNS so it should be easy to have something polling that queue and respond in any way necessary.

I'm imagining an alert to the on-call team, and a soft shutdown until the on-call team can figure out the next step.

If it can save a few thousand dollars, it's worth it. Each business must make their own estimate of course.

> AWS has budget alerts right?

Nope. Not in real time, they don't.

You can significantly overspend before the warning comes in.

Yep, for a static site you can throw nginx on some VPS for $10 a year and it'll handle a decent amount of traffic.

in other words, "if you're thinking of using netlify, don't".

true. I have a 9€/mo vps at Contabo for my blog and once boasted on HN that my small VPS is able to handle reddit/hn hugs which one user seemed to take personally and they started a DDOS against my VPS.

I only realized this after Contabo contacted me and said the traffic is so high that other clients service is also degraded and they will have to take my VPS down if its much longer (which was understandable). Gladly the ddos stopped soon.

But never was there any talk about any cost, they were very supportive

Even then Cloudflare forward proxy capable of real ddos handling wouldn't cost you $25 per month, some 0.5% of the 95% discounted bill.

But hey - just think about how much you saved on Netlify! Composable!

To some extent, that answer is fair enough, assuming they make this clear up front. If their service is "we'll keep your site up no matter what, for a price" that's a fine service to offer. It's not what the vast majority of people want, of course.

If their advertising is targeted to small businesses and individuals who could never afford this type of service, they could be guilty of false advertising, at least morally guilty. I haven't seen their marketing so I wouldn't want to say.

Their marketing is very much like this. It’s completely misleading. They are definitively not selling “keep it up at all costs, money no object”

This rings so true!

I've dealt with Netlify's support [1], and one of their CS heads was incredibly rude to me and blamed me for the problem they created.

[1] https://news.ycombinator.com/item?id=35610956

“Stop dressing so sexy if you don’t like the attention” is the vibe I got.

I don't fully understand Netlify, but it seems though it tries to be a one-stop solution for everything it doesn't have to be - you could put free Cloudflare in front of it and probably mitigate this kind of thing?

https://docs.netlify.com/domains-https/custom-domains/config...

Reply on news.ycombinator.com