CDNs and Centrality | Hacker News Reader

CDNs and Centrality

potaroo.net

60 pointsrajnathani23 comments

This article slightly misunderstands key details from the akamai blog post[1].

>Akamai experienced an outage for one of its Prolexic DDoS services (Routed 3.0) starting at 4:20 AM UTC.

Prolexic "Routed 3.0" is not a CDN product, and is generally separate from Akamai's other services (CDN, DNS). It is a layer 3 DDOS scrubbing product that works by advertising BGP routes for customer IP addresses filtering out attack traffic, and then routes the traffic to the real destination over GRE tunnels.[2]

This explains why the customers effected were only customers who are "always on" like banks, who are wary of using a layer-7 CDNs because they don't want someone else terminating their TLS connections. Prolexic doesn't make your site faster, it makes it slower.

In AWS terms: the issue was with "Shield" not with "Cloudfront". If Akamai's actual CDN went down, the impact would have been larger than Fastly's outage.

[1]: https://blogs.akamai.com/2021/06/akamai-provides-prolexic-dd...

[2]: https://www.akamai.com/us/en/multimedia/documents/product-br...

Does anyone adopt akamai for greenfield nowadays? Seems a dated product, it's either fastly or cloudflare or CloudFront it seems nowadays

Akamai’s edge performance is still first-class but I do think QUIC/http 2/http 3 have significantly reduced the benefit over more centralized CDN offerings.

It is a fully integrated core service built on the Akamai edge platform when you are buying and implementing it, and then a tiny little no-nothing "gosh are people still using that?!" in the corner when it goes down.

You also might want to take a look at who Geoff is before claiming that he is misunderstanding how a routing product works. https://www.internethalloffame.org/inductees/geoff-huston

I'm sure he understands how it works, but IMO calling Prolexic a CDN is wrong.

I don't agree that the network effect in CDNs is so huge. Instead of consolidating, I think the CDN market is slowly fracturing. 10 years ago, there was only one workable CDN for most: Akamai (founded 1998). Now there is Cloudfront (launched 2008), Cloudflare (founded 2009), Fastly (founded 2011) and a long tail of others. It takes some years for a new CDN to sign big corporate contracts but the trend seems to be toward more CDNs and a number have been founded recently.

It is easier than ever for them to get started because they can begin by piggybacking on one of the existing CDNs. Eventually they will build their own network of POPs (local caches). I also don't think you need as many POPs now to build a saleable CDN as in the past - there is some fraction of the market for which as low as 5 POPs might be ok.

Conversely, the usage of CDNs has increased since the 'Akamai-only' days. Case in point: The vast population of WordPress and other web applications wouldn't generally use Akamai in the 2000s, however, today CDN usage is more commonplace.

> It is easier than ever for them to get started because they can begin by piggybacking on one of the existing CDNs. Eventually they will build their own network of POPs

This increases centrality, as the point of failure is the parent CDN.

Agreed - the usage has risen hugely but that doesn't mean that it won't continue to fracture. The barriers to entry in running a CDN are not enormous and lots of startups are able to enter.

> This increases centrality, as the point of failure is the parent CDN.

Only for a while, as they are starting up

Creating POPs is not as trivial as creating a Heroku or Render like service on top of public clouds. While there are EC2s and public cloud VMs in various geographically distributed locations around the world, the instances in the actual POP regions are usually inaccessible.

Example:

- AWS has 25 regions [0]

- AWS CloudFront has 225 POP regions [1]

Partnering with Equinix/etc. and other ISPs to establish more POPs is an expensive and time-consuming process. Unless the CDN has a good edge-close-to-cities presence and a high number of those, users will generally not see value in the CDN.

[0] https://aws.amazon.com/about-aws/global-infrastructure/

[1] https://aws.amazon.com/cloudfront/features/?nc=sn&loc=2&what...

> Creating POPs is not as trivial as creating a Heroku or Render like service on top of public clouds.

I didn't claim that it was.

> Unless the CDN has a good edge-close-to-cities presence and a high number of those, users will generally not see value in the CDN.

A home truth is that most CDN users don't give a monkeys about per-city POPs. If you just had one POP per continent that would probably be sufficient to cut latency by ~75% for cache hits. Many real world CDNs have <50 POPs (eg CDN77, which has 33) and some have <25 (eg BunnyCDN sells a package with just 8 POPs). These businesses have already existed for some years and have a real customer bases.

Sorry it was ambiguous. By “users”, we mean the business buyers of the CDN service. Eg: As a developer, I don’t know where exactly my users are, if thus I was spending the same, then I would rationally prefer a CDN which has more POPs than one which has fewer.

It wasn't ambiguous, I meant buyers.

There may be more CDNs but most of them aren't that great…

CloudFront has HTTP/2 prioritisation, and concatenation issues

Many of the others build on nginx which also has HTTP/2 prioritisation issues

Akamai, Cloudflare & Fastly seem to be the only CDNs that are seriously investing in their infrastructure and their platform — all use custom edge server software (yes some of it forked from open source), have built in image optimisation services, have ability to run workloads on the edge etc.

Then there's the peering arrangements that have to be negotiated (and paid for if you're not pushing significant traffic)

By its very nature only a few players will be able to offer (effective) DDoS protection. They are in a constant arms race for bandwidth with botnets, and with the internet of shit botnets aren't getting smaller anytime soon.

There does seems to be less diversity across the board as services like DNS, SSL certs, etc, all converge into a few big players.

Not to say there aren't choices and smaller providers. Just that a higher percentage of the business is going to the biggest players than in years past.

The fact that the stock price of both akamai and fastly increased after these incidents makes it seem like these incidents don’t matter. I wonder what’s the threshold at which market sentiment flips from “everybody makes mistakes” to “these providers are incompetent and will lose customers”.

> I wonder what’s the threshold at which market sentiment flips from “everybody makes mistakes” to “these providers are incompetent and will lose customers”

When some internal document leaks that reveals "internal struggles" or some master plan, akin to the Panama Papers. The problem is, the Panama Papers are just about forgotten today.

Stock price is not a great place to look at for these incidents, I believe. It includes people who think the incident means the company is overvalued, people playing against the first, and people who think it matters in general but not for the company survival/customers long term.

In the end, the views may balance out, or even the "I'll make a quick buck shorting / buying on bad news" sentiment may actually raise the price.

(In other words, if "everyone knows" where the price will move today, the price won't move unless something is really affected long-term)

I think in fastly's case they were not very well known and suddenly the market became aware that they were actually a big deal with lots of big customers. Probably a one-off event.

Note: I am a fastly customer, reasonably happy.

> makes it seem like these incidents don’t matter

They don't really matter, to be honest.

If anything, they give insight into the growth of companies, showing how many key players use them. Fastly goes down and you're like damn, all these people using Fastly... I had no idea.

Stock price will dip if the market thinks the company is not doing well or projected to not do well. Nothing else is in consideration.

Stock prices do not go down when interest rates are near zero; simply because investors have no choice but to buy stocks.

If the Fed fund rates were 2% or 5%, FSLY and AKAM would look very differently.

For me, the threshold is when they actually start losing customers (or more specifically, losing revenue).

Reply on news.ycombinator.com