Cloudflare on the edge
stratechery.com
stratechery.com
Re: Edge: AWS has answers in Lambda at Edge (PoPs), Wavelength (5G Edge), and Local Zones (metro DCs); whilst Google has similar arrangements through its Anthos product line, if I'm not mistaken. Though, where Cloudflare shines is they architect their products to deploy and run from all locations around the world.
Cloudflare also positions its ("global") products as being the simplest of all cloud providers to use. We stand to see how long they can remain as simple once they start ramping up on newer product lines and eventually have to support different integration points among them.
The article points out that Matthew Prince, being a HBS graduate, quotes Clayton Christensen in his TechCrunch Disrupt Q&A; and from what little I gather, Andy Jassy comes from the same school of thought, as well [0].
I'm looking forward to more innovation from Cloudflare [1]. I've slowly moved all workloads that Cloudflare Edge could support out from AWS to it; only regret is AWS credits lay waste especially given we are pre-revenue. That said, Cloudflare's (limited) offerings are a better choice if you're a small engineering shop and couldn't be bothered with a gazillion bill items, IAM rules, and VPCs.
[1] ...and new-age cloud computing companies like railway.app, replit.com, darklang.com, stackpath.com, deno.land, vercel.com, fly.io to name a few.
> "Sure. I mean, for us, that’s easy. And then we can provide that to our customers as a function of what we’re doing. But I think that if you could say, German rules don’t extend beyond Germany and French rules don’t extend beyond France and Chinese rules don’t extend beyond China and that you have some human rights floor that’s in there."
I'd be curious about that human rights floor. It's one thing to follow laws in a western democratic country with elections and rule of law, but it's another to do so in countries without elected governments or rule of law. It sounds like they're thinking about this since he explicitly mentioned the human rights floor, so that's good at least.
> "Right. But given the nature of the internet, isn’t that the whole problem? Because, anyone in Germany can go to any website outside of Germany.
> "That’s the way it used to be, I’m not sure that’s going to be the way it’s going to be in the future."
This makes me sad, I hope we don't end up in a highly nationalized intranet style future but it does seem like things are trending that way (unfortunately).
If the internet is going to remain an open and fully connected global network, it’s absolutely necessary that internet entities don’t need to worry about the laws of some country halfway around the world.
The alternative is, at best, every website being IP-locked to a small set of countries. Either self-censored to countries that sites feel legally safe exposing themselves to, or censored by governments at transnational network boundaries.
For me, the DCMA is the laws of a country halfway around the world, that the US tries damn hard to enforce on other countries.
In a simial vein I can see myself being in favor of some basic privacy rights applicable worldwide, too.
Being accountable to some level of democratic governance is not the bane of free speech and capitalism, it is the foundation of a truly free market that has space for newcomers to innovate and compete, as well as citizens to have their rights protected.
The national democracies have created the legal and economic infrastructure for companies to go international, it's silly to think that without that basis companies could even exist at all. They need protection by the rule of law as much as citizens. The free market is not a jungle - for an open internet for example we really need some form of net neutrality.
I think of lot of EU citizens feel we also need some form of GDPR that is actually respected, and the US is actually the problem hindering an open, fully connected internet and global network here.
If democratically elected governments aren't supposed to regulate corporations then who should?
I'm in Australia. If I buy ads on Google I believe I'm actually dealing with Google Ireland, because the money goes there.
But they have servers in Australia. But their headquarters is in the US.
The EU approach (~"oh, you want to move where you measure your revenue around? Ok, we will look at global revenue") seems like a logical response to the way multinationals use corporate structure to avoid tax.
No one elected Zuckerberg to be the global monarch. I don’t want him making the laws that govern global society.
Why should foreign companies be entitled to enter a country and break said countries laws? Is it your position that we should do anyway with the notion of sovereignty and the rule of law all together?
So, to be clear, you are explicitly arguing for the case where the internet should be siloed on a country-by-country basis unless participants are willing and able to establish a legal business presence in every country they’re accessible from? That’s what “doing business” means on the internet.
> Why should foreign companies be entitled to enter a country and break said countries laws?
Sending packets over the internet isn’t “entering a country”.
The gap here for me is that AWS, GCP, and the like are perfectly equipped to build infra "from the edge in" as well. It's just not an advantage for Cloudflare -- I'd actually argue that AWS has the comparative advantage in building "from the edge in" given the clip at which they can turn out datacenters.
With the new Outpost model as well, I don't see too many hurdles for AWS to just start deploying their whole feature set in any old colo, in any old country, in a matter of months. They do have to set their minds to it though. At the very least, they should definitely have their storage services in every country with actual data locality laws.
I'm sure I'm missing something, but on the infra side I don't see what Cloudflare is better at in this regard. I'd also argue that AWS storage migration is just as good or better than Cloudflare's offering here. I won't comment on how users perceive/value each service because I don't use either much.
What I definitely DO agree with, is that infra "from the edge in" is a major threat to existing public cloud business models. At the point where you have your whole stack portable from colo to colo (or, gasp, as a dAPP), it starts to sound more like public cloud as a library, rather than as a centralized managed service.
Open source communities, unbundlers, and new-age cloud companies together will have a field day building 'library' versions of common cloud services and I think Cloudflare will pair nicely at that point. The question is when that becomes a reality.
(That said, I think given their experience deploying to China, meeting GDPR, etc, and their focus on whatever it takes to make money, AWS and others should have no problem figuring out a solution to local regulations if that's what it ends up coming down to. That's far more likely than enterprises deciding to restructure everything they do around Cloudflare Workers, or Cloudflare becoming a full cloud platform.)
Outpost may be local zones today (needs fact check), but maybe a better question is what is the minimum service threshold for AWS to deploy a new "global zone". Considering data locality is the topic at-hand, I would surmise it's not that much and that Big Cloud can handily deploy new global zones at the rate that these new regulations crop up.
Totally separate topic -- but my other thought is these laws are totally at odds with the decentralization trend. Not sure how FileCoin, Sia, Storj et al are thinking about this... maybe building the decentralized storage _protocol_ is the escape hatch, but maybe not.
You’re clearly more versed with AWS than I am but I think that if you compare them at this level, you could argue that all the big cloud providers could easily compete with Cloudflare and crowd out their proposition.
But this possibly overlooks the (in my view) very unique execution style of a Cloudflare compared to “Big Cloud”.
I do worry about their ever growing reach across vast portions of the internet, but, I keep coming back to Cloudflare and taking up more of their feature set because they make me so damn productive, and unlike all the other cloud providers, the learning curve is very minimal and their products are a pleasure to use.
I have huge respect for AWS and Azure, and use them both for production workloads where needed, but I’m weary of their complexity and bloat. Cloudflare is an absolute breath of fresh air in comparison.
My only criticism of CF’s execution is that it can seem a bit “scatter gun” sometimes. Eg Cloudflare Registrar still doesn’t support .co.uk domains years after its launch - which somewhat undermines that offering. They sometimes launch things and then seem to neglect them and start working on something else.
To end on a bit of a tangent: the one thing I really wish they would turn their creative minds to is reinventing the whole CAPTCHA concept. Given their reliance on captcha and what an utterly fucking awful experience it is to try and identify all the buses in 9 grainy low res photos, I think it’s about time CF invented something better!
I would definitely argue the first point here, but not the point about crowding out. You said it yourself, there's totally room for a Cloudflare to win on UX/DX, and other areas like cost. This is the strategy the CEO mentions when he's quoting Christensen, and coming up from the bottom to compete with the big dogs. It's also why AWS will make more in a day than Cloudflare does in a year though, for now..
Panelist: "Would your vision eventually, this disruption, take over internal teams working on these problems."
CEO: "Our vision is we're going to power the Internet. [Dead serious face and stare]."
Video in the article. Amazing vision and execution.
Shows the immaturity in the analysis of stratechery in my opinion.
I agree the journey probably wasn’t as 0 or 1 as my comment makes it seem.
But in all seriousness, it’s okay to relax a little, not to be negative, and concede some things to be inspired.
:-p
But it really was how we thought about what we were doing from day one. Our litmus test was always: “If we ran the Internet, would this be the right decision?” Asked that for every technical, business, and policy/legal question we faced. Seemed absurd when it was 8 of us over a nail salon in Palo Alto, but led to a lot of good decisions and long-term thinking.
I just found about Durable Objects and can't wait to try them on a new project for a client that has a use case like: few days per year has a huge traffic spike (tens of thousands concurrent users) while the rest of the year is pretty much dead. I think DOs could be a perfect fit for this.
Cheers and let's hope for another 10 great years, :D.
That’s the way it used to be, I’m not sure that’s going to be the way it’s going to be in the future.
Uh oh. He might be right.
Sort of, but partly because those bigger companies are often the ones writing the laws. Microsoft, Google, Amazon, etc. operate "public policy teams", whose goal is to push legislators to enshrine their business practices as law.
Don't get me wrong, I'm nowhere near an economic laissez-fair libertarian, and I believe that strong regulation is necessary in many markets and industries. However, I'm also very wary of well-meaning but overly complex rules which do little but add a huge barrier to competition against the big players.
Once they add enough to become a full fledged cloud provider that an enterprise can lift and shift their SAP and Oracle bare metal servers and whatever else onto with guaranteed performance and availability SLAs, then they're in the same boat. Until they do that, they'll continue to be a niche player for things that fit those two and a half use cases. It's not like enterprises are going to rewrite all their enterprise apps around Workers.
I think this is just a matter of people getting tired of the self-driving cars story, and a reach for some other reason to pump up share values.
The tooling, documentation and language integration is rather awkward and cumbersome at the moment.
Sparse documentation, language tooling that seems very half baked, workflows don't feel intuitive.
But there is definitely potential.
> But there is definitely potential.
I agree! It's something I'm watching with eager eyes, but don't feel like is the first option I'd turn to when making a technology choice.
Another technology that gives me this feeling is FaunaDB [0].
[0]: https://fauna.com/
Here is something that can help with Cloudflare horrible tooling: Use wasm/Rust. You'll be able to compile and run most of your code locally, and then when you deploy you have more odds of your code not bugging on the cloud.
I would love to have a local runtime so I could iterate faster.
It makes it really difficult to deal with crime, cyber warfare, and illicit content. We need "something."
People are starting to realize what happens when a central entity can just "cancel" you and as that keeps happening more often (ex: youtubers getting de-monetized, people getting locked out of their google accounts, Trump twitter cancellation, etc), people will be pushed to these un-censorable decentralized alternatives.
If you're using TCP, bandwidth depends on latency, and surely no one thinks bandwidth isn't important? This totally does matter in the hundreds-of-milliseconds range, you can use one of the online calculators to confirm this.
The other thing is that live video/audio isn't niche at all. Cloudflare could prioritize latency-sensitive traffic like that within its network, and deliver it as close to the users as possible on their respective ends.
Am I missing something?
In many applications, there are certain expectations of performance around certain actions, sure. That said, if things go long the user may get annoyed but they won't be deterred. But in some cases, having anything but the highest regard for latency is not only a deterrence, it's basically completely unacceptable to the user.
Sure, you want your blog to load fast. But if your blog takes 300ms or 900ms very few people are going to notice the difference despite the 3x disparity. You're unlikely to get a nastygram on Twitter about your blog being unusable. Hell, even websites that take greater-than-one seconds to load are routinely not taken to task by the average user because, quite frankly, while fast is nice it's not a need.
But apply that to a competitive, real-time game and if your service takes 70ms RTT for one of the roughly several hundreds of thousands of commands issued to it over the course of a 20 minute span you're performing about average. At 250ms - dramatically less than the kind of latency bound available to a blog post - you're rapidly approaching "completely unplayable" and going to get nasty responses.
I think talking about the modern web in milliseconds doesn’t represent the way it really is. Based on my experience everything is measured in seconds and UIs are awful compared to what we had 10-20 years ago.
It’s like the old green screens that adept users could outperform, but with better graphics and no buffer/command queue.
Still, empirically, latency has a huge impact on throughput. Part of the reason might be that many transfers are actually too small to take full advantage of window scaling? Maybe there are other factors at play as well that I'm not aware of.
Check out for example this speed measurement tool from AWS [0], which shows a really drastic influence of physical distance on transfer speeds (particularly for the small files where I get >10x differences, but even at 5 MB I see >3x differences between a close-by zone and those roughly on the other side of the planet). That happens even with their "global accelerator" service which gives them a lot of control over the transfer parameters.
It’s great for inserting data into a page, because it runs on super fast connections, and means the data is already in the page when it gets to users, there’s no waiting for the browser to fetch and hydrate data into the page, so the UX is awesome.
And security is great too because you don’t need to store any access keys in the browser, instead have these in env variables in Cloudflare.
I'm interested to hear from anyone from fly.io on this - if Inunderstand it their approach is very amenable to this as well - at the simplest level all my app needs is an environment setting telling me which data centre i happen to be running in. then i can start doing whatever regulations I need to.
Instead of pitching a fit at Cloudflare, encourage and promote positive views of Tor-based browsing, privacy-respecting browser settings, etc. There's an unfortunate reality that people want those Cloudflare settings because they stop a lot of malicious traffic, while allowing the vast majority of legitimate traffic. As we move the window of legitimate traffic to be more privacy-respecting, you'll see the boundaries of such behaviors are forced to adjust.
I don't know if the person to whom you are replying is using Tor, but I'm definitely not and I also despise how Cloudflare--or their customers, but I have no way of differentiating the two--reacts to various things.
My home ISP is a very small outfit that mainly serves apartment buildings in my area. I don't know why, but every three or so weeks, Cloudflare decides that my ISP is a massive pile of risk and starts shunting almost every request I make through their "analysing your browser please be patient" screen. It's nothing specific to my computer because if I flip on a tunnel through my server in a colocation service in the same city as me, I get zero prompts.
I've worked with my ISP and they've basically given me free reign to assign myself whatever public IP I want out of their /22 (as long as it's not in use by another customer, of course) and, during these times, any IP I choose gets the same response from Cloudflare.
It's the same lack of transparency and being able to say "hey, what gives" that applies to spam filtering by the big e-mail providers or getting dropped into the "scam likely" bucket for mobile phone calls. I understand the need to filter and screen and protect but that doesn't mean I have to like it when it splashes onto me.
Does Cloudflare provide something to say "Is my IP considered malicious?" and a way to ask/appeal about any bans?
This.
If your ISP isn't keeping track of which customer is using which IP addresses and handling complaints sent to their abuse@ etc, then their /22 is likely to have poor reputation.
[EDIT] though I'm not disputing that it would feel shitty and unfair, for a normal user, to be stuck dealing with CAPTCHAs because of that.
To be clear, I am the only person (so far as I know) who has the ability to do this. Subscribers are ordinarily segmented by VLAN and the DHCP server hands out a single, predefined IP address on a /30. If a subscriber moves between apartments, their IP goes with them (the VLAN is simply relocated to the new physical port in the new apartment). Which customer has which IP in which apartment is all documented in their customer management system.
What they've permitted me to do, as part of testing, is to manually assign myself an IP by removing the VLAN and DHCP restriction on my specific port. If I try to this same stunt in a friend's apartment, on the same ISP, it does not work.
[of course captchas do nothing against targeted attacks]
1: https://blog.cloudflare.com/cloudflare-supports-privacy-pass...
As long as we suck it slowly, they won't notice!
This only happens to me very occasionally (say, a couple times a year tops)
As a developer, I love their products.
I run a fairly decent sized website (perhaps not as big as Hacker News but comparable in scope and magnitude) and I'm frequently analysing traffic samples to determine ways to profile abusive behaviour. Traffic from TOR is usually in excess of 95% robotic and malicious. For me it's just a waste of time to allow any of it. (But I do, so long as your browser already has a valid authentication cookie generated outside of TOR.)
I don't like censorship, but I have a hard time criticizing those actions.
IBM, VW, Deutsche Bank, etc.
We are in a hyper-political environment, where your company is going to be judged by one group or another for any action or inaction. So if you're protecting far right nuts, your left side is going to drop you. In a very real sense, protecting 8chan or Daily Stormer is a company deciding to solely make itself palatable to supporters of those sites, at the expense of basically... all your other customers.
No company that wants to be successful is going to stand in defense of a customer or two that's known as a source of terrorism and violence.
Who are you thinking of when you talk about censorship?
Given that Prince studied law in Harvard he is way more knowledgeable and aware than average CEO.