What are the economics that Amazon and other providers have egress fees and R2 doesn't? Is it acting as a loss leader or does this model still make money for CloudFlare?
Amazon uses $/gb as a price gouging mechanism and also a QoS constraint. Every bit you send through their pipe is basically printing money for them, but they don't want to give you a reserved fraction of the pipe because then other people can't push their bits through that fraction. So they get the most efficient utilization by charging for the stuff you send through it, ripping everybody off equally.
Also, this way it's not cost effective to build a competitor to Amazon (or any bandwidth intensive business like a CDN or VPN) on top of Amazon itself. You fundamentally need to charge more by adding a layer of virtualization, which means "PaaS" companies built on Amazon are never a threat to AWS and actually symbiotically grow the revenue of the ecosystem by passing the price gouging onto their own customers.
It is pretty much exclusively to lock you into their services. It heavily impacts multi-cloud and outside of AWS service decisions when your data lives in AWS and is taxed at 5-9 cents a GB to come out. We have settled for inferior AWS solutions at times because the cost of moving things out is prohibitive (IE AWS Backup vs other providers)
To them "that's just what bandwidth costs" but anyone who's worked with this stuff (sounds like you and I both) can do the quick math and see what kind of money printing machine this scheme is.
Some people want to host a lot of warez and pirate movies and stuff but that doesn't monetize very well per GB consumed so pricing bandwidth high means those people never show up, thus saving a lot of trouble for AWS.
I remember when salesforce.com announced a service that would let you serve up web pages out of their database, it was priced crazy high (100-1000x too much) from the viewpoint of "I want to run a blog on this service" but for someone who wanted to put in a form to collect data from customers it was totally affordable. Salesforce knew which customers it wanted and priced accordingly.
The issue isn’t charging for egress, but charging excessively.
First a lot of these roads are 'free' and yet you're still being charged for it. If two large networks come to an agreement then they connect the two networks (ie build that road), but no money changes hands.
Second if there is a paid peering agreement in place (ie say AWS had a cost to push your data out), that still wouldn't be billed to them in the way they're charging you. Instead they'd be paying for the rate of traffic at something like the 95th percentile of the max. This means that you could download a petabyte of data from them when the pipe isn't busy and cost them nothing, or you could download a gigabyte when it's busy and push up the costs.
Cloudflare has huge ingress, because they need it to protect sites against DDOS.
They basically already pay for their R2 bandwidth ( = egress) because of that.
Additionally, with their SDN ( software defined networking) they can fine-tune some of the Data-Flow/bandwidth too.
That's how I understood it, fyi.
Some more info could be found when they started ( or co-founded, not sure) the bandwidth alliance.
Eg.
A CDN keeps the data nearby, reducing the need to pay egress to the big bandwidth providers.
( not an expert though)
You setup your website and preferably DON'T have it talk to anyone other than the CDN.
You then point your DNS to wherever the CDN tells you to. (Or let them take over DNS. Depends on the provider.)
The CDN then will fetch data from your site and cache it, as needed.
Your site is the "origin", in CDN speak.
If Cloudflare can move the origin within their network, there is huge cost savings and reliability increases there. This is game changing stuff. Do not under estimate it.
In the CDN case Cloudflare has to fetch it from the origin, cache (store) it anyway, and then egress it. By charging for R2 they're moving that cost center to a profit one.
The original post also includes a link to a more recent Cloudflare blog post on AWS bandwidth charges: https://blog.cloudflare.com/aws-egregious-egress/
I almost wish they had some kind of sustainable usage-based charge that was much lower than AWS.
Feel free to tell me why I’m wrong! I’d love to jump onboard - it just seems too good to be true in the long-term.
There probably needs to be an abuse prevention rate limit (and probably is), but it's not quite as crazy as it sounds to just rely on their CDN bandwidth sharing policies instead of charging.
I do think there are “soft limits” in place like you say - it’s just my personal preference to have documented limits (or pay fairly for what you use). IMO it helps stop abuse, and prevents billing surprises for legitimate heavy use-cases.
But that's really no different from the guarantee you get from most CDN services. If you're using cloudflare in front of S3, for example, you'll end up with the same behavior.
But in my mind it’s also comforting that something like Cloudfront has a long-term sustainable model (I should also add with fewer strings attached like hosting video).
I do think the prices ant AWS are too high, but it discourages bad actors from filling up the shared pipes. ISPs are sometimes a classic example of what happens when a link is over subscribed.
Cloudflare’s “soft limits” are also somewhat of a dark pattern if you ask me. I like to know exactly how much something will cost, and it’s really hard to figure out with Cloudflare if you’re a high-traffic source. Do I hit the “soft limits,” or not? It’s really hard to say with their current model.
FWIW, I think Cloudflare is a great product right now - I am just skeptical they can keep it up forever.
The customer (because they are lazy, don't know better, aren't capable of, or all three) opts in to use various "convenient" CSP "services". These services could look convenient (and are always pretty to extremely expensive), they quickly becomes an integral part of the customer's badly architected "system".
The end result is complete vendor-lockin, the inability of the poor (stupid) user to leave and the continued gang rape of their bank account (also via additional, incompetent developer and devops "resources").
Throw in average modern "devops" who are hired to handle this. They aren't like the sysadmin of yesteryear, they no longer have experience with, or understand the bits and bytes. They are glorified UI clickers and YAML editors, they even lack any reasonable system level debugging skills. For every problem they encounter they first immediately run to google in search for answers.
In addition, I would argue that CSPs are a huge, huge waste of computing, space and power resources, because their systems completely encourage people to just do things, without understanding what they are doing, screw the consequences and just pay.
Result, the business suffers greatly (on so many levels), the CSP wins big and continues winning.
What happens here is that a system, if designed right from the get go, could have been run on a SINGLE, modern, high end, well positioned and connected server to the Internet, is now replaced with tens to hundreds of "instances" and random assorted CSP provided services -- what a colossal waste.
Books can be written on negligence, lack of understanding, utter tech stupidity and ultimately the costs which are absurd.
It's also a huge waste of human effort managing the complexity introduced by the cloud provider's arbitrary bullshit.
At this point multiple generations of engineers have little understanding of underlying layers of technology, having only really learned how to use cloud services. No TCP/IP, no UNIX, just a bit of bash and a ton of AWS.
Cloud providers do hide most of the low level complexity, which could be seen as a benefit (at least that seems to be what's touted as a main benefit, along with instant scalability.) Unfortunately they replace all of that with more arbitrary complexity which is ultimately (in my opinion, at least) a much bigger burden than the fundamental complexity that is abstracted away.
Of course you can (like Snap) but it's a MASSIVE engineering effort and initial expense.
It's an option that's often been attractive if/when you didn't want the hassle of building out something that could provide S3 level durability yourself. But with more/cheaper S3 competitors it's becoming a significantly less attractive option.
I used the storage box from hetzner before but they only had 1TB or 5TB (and higher) choices so I had to pay for 5TB (€12 per month) without using most of it. Having rsync support was nice but rclone works fine with S3.