A bit of math around Cloudflare's R2 pricing model
twitter.com
twitter.com
“R2 will zero-rate infrequent storage operations under a threshold — currently planned to be in the single digit requests per second range. Above this range, R2 will charge significantly less per-operation than the major providers. Our object storage will be extremely inexpensive for infrequent access and yet capable of and cheaper than major incumbent providers at scale.”
Source: https://blog.cloudflare.com/introducing-r2-object-storage/
The lack of 9c per GB to egress is the killer.
I can rent unlimited 250mbit servers for 14 euros a month from Scaleway. It is truly unmetered; I've been a customer for years and do about 40 terabytes of egress a month on this box.
AWS would charge me $3800 for the egress alone (using the calculator). That is certainly almost all pure margin for them, when Scaleway will happily let me egress 40TB a month for years on 13.99 euro per month (plus I get a low-spec bare metal server at this price too).
https://aws.amazon.com/s3/pricing/ https://www.backblaze.com/b2/b2-transactions-price.html
But for write once, read a handful of times (e.g. a webhook system that basically is write once, read once of <32KB objects with long history for either redelivery attempts or replaying the entire stream for a point of time), request prices end up being the biggest cost.
Free per object requests is super game-able. Want 1000 free requests per second, just rotate through 100 identical objects.
1,000 reads per second is 2.6 billion per month. At S3 prices, that wound cost around $1,040.
100x storage is pretty minimal to achieve this. For example:
100x 128KiB = $0.000192/month
100x 10MiB = $0.015/month
100x 1GiB = $1.50/month
Yes. While there is a pretty hefty up-front cost to all the hardware and fiber interconnects AWS makes between servers, between AZs, and to the internet (probably in the millions per-dc including labor), the margin they charge per-gb is insanely high to the point where they likely made all of it back within the first day it was installed, with any extra usage in the years after being pure margin.
If we're looking only at egress, we can even find cheaper alternatives. OVH, for example, offers VPS' with 500Mbps unlimited for 10 euros a month, 1Gbps for 20 euros, etc.
https://us.ovhcloud.com/order/vps/?v=3#/vps/build?selection=...
Though I don't know how well it compares.
Edit: pricing: https://us.ovhcloud.com/public-cloud/prices/#containers-and-...
Not really...
https://www.hostgator.com/tos/vps-tos
VPS accounts may not:
exceed a 15 minute load average greater than two (2) times the amount of CPU cores given.
run public IRCd's or malicious bots.
run any type of BitTorrent client or tracker that links to or downloads illegal content.
use an Open/Public proxy, or utilize a proxy to access illegal/malicious content.
se I/O intensive applications which adversely affect normal server operations.
I do see they have a bit of a vague way of getting out of hosting you if you're really being excessive on the network throughput. It largely sounds like as long as your network usage isn't causing other customer impacts its still unlimited. I imagine even OVH or any other hosting service will have similar terminology along the lines of any usage that negatively impacts other customers is prohibited.
https://www.hostgator.com/tos/acceptable-use-policy
You may not consume excessive amounts of server or network resources or use the Services in any way which results in server performance issues or which interrupts service for other customers. Prohibited activities that contribute to excessive use, include without limitation:
Hosting or linking to an anonymous proxy server;
Operating a file sharing site;
Hosting scripts or processes that adversely impact our systems; or
Utilizing software that interfaces with an Internet Relay Chat (IRC) network.
It's not unlimited, but 20TB is more than enough for the most use cases.
It’s great that they are happy to have a free tier, but <10RPS is only <777k per day constant. Great for small uses, but pretty easy to hit with even medium object store use-cases.
My current guess would be somewhere around S3’s $5/million writes and $0.40/million reads (with maybe a 20% discount like they’ve done with other products). Hope I’m wrong and it’s a lot lower though.
I’m probably in the minority, but I want to use an object store like this as a key/value store for <128KiB objects (write once, read a handful of times and then expire in 14-90 days).
I’d gladly pay egress, but usually the request prices (for S3 or DynamoDB) become pretty cost-prohibitive. Even services like Wasabi that, on the face of it look perfect, end up being unsuitable because the T&Cs limit maximum egress to total monthly storage volume (making the write once, read more than once for every short-lived object pretty difficult).
I’d gladly pay for a service with both “reasonable” egress and “reasonable” request prices instead of everyone trying to weight their service to one or the other.
Workers KV is also $5/million writes and $0.50/million reads[1] (more expensive than S3 which is $0.40/million reads) and often DynamoDB (which is $1.25/million 1KB writes and $0.25/million reads scaling down to 1/7th that if you use provisioned instead of on-demand).
Workers KV is also eventually-consistent with no guarantee of read-after-write, which is a pretty big limitation compared to alternatives (S3 even has immediately-consistent list operations now after write).
Durable Objects Storage is $1/million writes at 4KB and $0.20/million reads[2], which is pretty good. But you can also only access it via a Durable Object which limits its usage (especially as neither Durable Objects or its storage are in a production state with final pricing yet) and means you need to also pay for the Durable Object runtime on top fo the Storage pricing whenever you want to access that data.
[1] https://developers.cloudflare.com/workers/platform/pricing#p...
Presumably with not-a-lot of code you could write a Durable Object to have an S3 compatible API...
Given the point of R2, the fact there's $0.045/GB ingress + egress data transfer (which has been recently reduced from $0.09) when using Durable Objects.
Given the original comment is basically about egress being too expensive therefore R2 is offering to for free, Durable Objects' Store isn't a particularly good alternative to just using S3 or DynamoDB unless you're already using Workers/Durable Objects.
DynamoDB writes are priced per WCU.
A WCU will give you 1KiB written to the DB.
In On Demand, WCUs cost you (depending on region) $1.25/million writes. For Provisioned capacity, you pay about 1/7th that (but need to deal with keeping the Provisioned capacity and auto-scaler at the correct level, so in practice you can maybe get this to about 25-35% the On Demand rate when you know what your usage looks like, with further discounts for actually reserving capacity).
If you're writing 32KiB objects, this uses 32x WCU ($40/million writes) and if these objects are even updated (e.g. to change a status or similar), the update writes the entire object again.
On a performance/architecture note, DynamoDB has a hard per-partition limit of 1,000 WCU (per second). Writing larger objects can very quickly get you to these limits which increase your architectural complexity by requiring you to shard access.
Although, in practice you'd never write large objects to DDB. S3 is _only_ $5/million writes (and $0.023/GB/month vs $0.25/GB/month for DDB). If you're writing even 3+ KB (especially if you expect to need to update an item in the future) it's usually more cost effective to store these blobs in S3.
But either way, you're still going to be paying at least $5/million write requests. For multi-MB blobs, this is a non-issue. But for a K/V store of smallish objects, I think it's pretty expensive.
The underlying tech can differ depending on object size, but as a user, I want it to all transparently look like one unified store.
_Maybe_ Firebase Store for certain scenarios, but it has its own tradeoffs too.
When I first saw this, my thought was it's highly misleading since you'd normally be fronting S3 with a CDN, so you'd only have to pay once to transfer a given resource every few days/weeks/months if the CDN is doing its job, and it can be served to millions of people at no or low cost (already been free on Cloudflare for a long time).
Then I came to my senses and realised there's probably orgs who don't know/care about that and they're actually paying these fees for every request.
Anyway, it's excellent to see this released. S3 is not just more expensive and tied to a single data centre. It's also bad DX, e.g. setting up permissions is way too complicated and the console is "brutalist" in both look and function. I hope this spurs some more competition in the area of storage.
The difference is still insane.
> Okay, then it’ll be 8.5¢ to 12¢ per GB depending upon where you happen to be downloading it from.
[1] https://twitter.com/QuinnyPig/status/1443135455763984384
Kudos Cloudflare! You won the Internet this week!
Some assets like logo's, headers, footers, css, fonts etc will improve greatly.
But the main content is highly long-tail. And has a very high cache-miss chance.
For 140 chars that's not going to matter. But all the images, videos, large source-codes, tarballs, builds, etc all add up. No CDN is going to help when a 30MB nightly build is only downloaded once ever. When a 2000line sourcefile is read maybe once a month. Or when a video in a tweet is seen by at most 2 followers ever.
For a single download of a large file, yes, it's probably not going to matter since you're talking a percentage of the total file transfer time but the story can be different for something like a web page where the client makes a single connection and reuses it for everything rather than having to establish connections to multiple servers, wait for window scaling to kick in, etc.
Many of those reasons will apply to long-tail content like GH of Twitter and offer benefits there.
I was talking about the most obvious benefit of a CDN: caching on edge-servers.
Do you have more detail about this? We cache a lot of image content with CF, though I haven't looked at cache effectiveness stats closely.
- IAM / Bucket Policies as mentioned
- S3 Block Public Access (yes this is the name of a distinct feature)
- S3 Access Points
The interactions between all four of the above need to be taken into account to determine the access a consumer has to a bucket or its objects.
I'm not taking a swing at IAM - it's a great system. My grudge is with AWS having disparate interacting security controls for different services (of which S3 is probably the worst offender).
To heave back onto the topic at hand, let's hope R2 can do better having the benefit of a clean slate to build off!
I can't remember the last time there was this much interest in a new product launch - which gives an indication to the market need for pricing changes on this front. However, AWS re:invent is right around the corner and I suspect this will cause an announcement or two to come from the AWS side.
Call me a pessimist, but I expect that will happen around the same time they make managed NAT gateways reasonably priced.
I'm going to say never. I'll likely be right for years before I am eventually wrong due to the sticker price not moving but inflation making it relatively affordable.
Go ahead AWS, prove me wrong!
Being a proxy is cool.
Kudos to all involved in this project! :)
They would also bring cloudflare in the spotlight and they already have the core products as a cloud hoster ( rdbms, nosql, serverless, apps, ... ).
Additionally, reducing eggres and making use of 3rd party products possible is not in their interest long term ( eg. The fork of elastic search).
What do you think they can do/announce?
Removing egress feed is not just about taking a massive hit to revenue. If it was, they might bite the bullet and hope that the lower prices would attract more customers in the long term. But bandwidth pricing is the moat around their castle. If it didn’t exist, Snowflake and others would leave. But more than that, new AWS services benefit from the captive audience choosing them by default. Without the moat, each service has to compete on its merits. AWS Lambda, for example would need to be better than CloudFlare Workers or fly.io containers. It won’t win customers like it does today just because it’s “free” bandwidth.
They won’t rush to announce anything that might affect the foundation of their multi billion dollar money printing machine. They’ll take their time and see how customers react. If any large customer threatens to leave, they’ll be offered credits to stay. But that information about what customers want and what they’re likely to do will help them come up with a strategy.
My contention, as I've commented here before, is that AWS is taking advantage of a new "cloud native" generation of developers, startup founders, C-suite, etc who have never purchased bandwidth/transit, run an AS (autonomous system), etc. As mentioned in the twitter thread Amazon is essentially charging 1998 prices for bandwidth. I wouldn't be surprised at all if many of their services are essentially loss-leaders and Amazon more than makes up for it with their ridiculous markup on bandwidth.
This "cloud native" generation can wrap their heads around instances, request rates, storage volume, K8s, containers, functions, etc but the pricing on bandwidth is very nebulous, notoriously difficult to estimate/control, and poorly understood. Without organizational experience and knowledge of the real bandwidth costs in 2021 the assumption is likely "well, that's just what bandwidth costs".
"Please submit the original source. If a post reports on something found on another site, submit the latter."
https://news.ycombinator.com/newsguidelines.html
* or rather, doesn't point to it? that's not good (edit: it does point to it now - sorry if I missed it earlier.)
Anyway, guidelines are guidelines, not cast iron rules. When the original site is hot garbage I'd much rather see an alternative.
I find the experience worse than Twitter's default web view with the odd reformatting and wasted space around tweets — compared to the native view, I see 5/7 as much text on a page — and most thread unrollers intentionally obscure the original link to keep you on their site so you have to be familiar with their UI or scrub the links to find it if you want to interact with the tweet (a common use for social media).
If there were a consensus I'd be happy to go with it and suspend the rule, but given that there's zero consensus, we should stick with the rule.
Something like https://threadreaderapp.com/thread/1443028078196711426.html I might advocate for. Better layout, plenty of links back, etc.
Imagine a competitor to YouTube. Storing tons of video content is expensive. Serving it to users is also expensive. Especially with those egress pricings. 1M * 1gb of egress costing 54k is completely undoable profitably unless that data is very valuable. Meanwhile <$1 is “put this MVP startup product on my credit card while applying to YCombinator” affordable.
If R2 could point to Backblaze instead of S3 as a compatibility layer (S3 compatible so should be easy), the you could have CHEAP redundancy and low costs without any egress fees.
I can think of several niche video sharing use cases not wel serviced by YouTube that just aren’t profitable enough to host so haven’t seen any startups succeed and I can totally see it now happening. Maybe I’ll try to make it happen…
Hosted.fm - free egress will mean no overage pricing needed!!
Inb4 Cloudflare announce a podcast hosting platform haha
R2 will be amazing for hosting computer vision datasets and models.
It is for a cloud hoster. But ML does not take advantage of cloudflare's competitive edge ( their SDN ).
In the same way, i don't think docker containers are an interesting/good fit ( i could be wrong on this though)
Edit: i was wrong...
Which is absolutely where CF are headed, as evidenced by R2.
Don’t forget that S3 was the first AWS product. It’s fitting that it’s also CF’s first real foray into cloud-land.
I thought that ai wasn't interesting on the edge.
Already really impressed with Cloudflare Sites and planning to adopt Images soon.
You guys deliver at a dizzying pace, it's truly incredible.
Full text search, please :)
Inverted indexes are almost always eventually consistent, and easy to integrate in a distributed context.
- Store documents into R2 or Durable Objects or KV or (Cloudflare future storage).
- Create inverted index files (similar to Lucene)
- Cache the inverted index files at the edge. Cloudflare’s bread and butter.
- Run query execution nodes at the edge. Deal with the occasional cache miss by proxying to a different region.
- Its impossible to deal with relevance or ranking for that many different use cases. Let the clients control every aspect of relevancy and ranking, e.g. Similar to the flexibility of ElasticSearch.
I'm wondering if cloudflare can add additional languages. The current languages on top of WASM is not so interesting to me. I'd love to try them out more seriously though.
Eg. A managed OpenFaas offering would be very interesting ( since it supports dotnet core, which in the current phase wouldn't be added i think)
But my interest would be satisfied enough if i know languages outside of WASM will/can be added in the future.
Edit: thanks for checking :)
Thing is, all cloud providers do own the pipes, so they just rent-seek on egress pricing. Cloudflare is disrupting that model.
That is the only avenue why I can see why paying for S3 is worth it. Sure egress may be free, but what's the point if I can't download a file faster than 1MB/s.
Linode, Digital Ocean and other hosting companies sell small VPSs with 4TB bandwidth for $10/month and still making profit. At places like OVH, Hetzner bandwith is virtually unlimited.
The average Linode customer is not using 136GB of bandwidth per host per day.
You or I can buy transit from e.g. he.net for ~$0.15/mbit/mo. 10Gbit transit @ $1500/mo will transfer 3PB/mo.
3PB egress on AWS = $157,491.11/mo.
And that’s before you take into account you can do settlement free peering at Internet exchanges.
I've seen this kind of marketing allot from cloud providers, and it always makes me wonder. At that point the probability that you're going to lose your data gets dominated by the probability of a rogue employee deleting your data, or of 3 simultaneous natural disasters destroying every warehouse your data was replicated to, or most likely someone gaining malicious access and deleting a bunch of stuff. Those are a little harder to quantify though.
https://cloud.google.com/vpc/network-pricing
https://cloud.google.com/storage/pricing#network-pricing
The only thing that is free is between nodes of the same zone using specifically the internal IPs.
> If your use case exceeds the guidelines of our free egress policy on a regular basis, we reserve the right to limit or suspend your service.
So store 1PB for an hour, pay for it for 3 months.
Also egress is not always free. Depends on how much you need the data to be in the free tier.
Source: already discussed
https://wasabi-support.zendesk.com/hc/en-us/articles/3600023...
They also have a minimum object size of 4K:
"If you use Wasabi to store files that are less than 4 kilobytes (KB) in size, you should be aware that Wasabi’s minimum file size from a charging perspective is 4 KB. You can store files smaller than 4 KB with Wasabi but (for example), if you store a 2 KB file with Wasabi, you will be charged as if it were a 4 KB file. This policy is comparable to minimum capacity charge per object policies in use by some AWS storage classes (for example, AWS S3 IA has a minimum capacity charge of 128 KB)."
THe one I really hate is that 90-day deleted file charge. If you upload a 1K file every day for a month, into the same filename, you get charged for:
4K the first day, plus 8K the next day, plus ... plus 360K the 90th day and every day after that if you keep uploading every day
They are charging 360x your actual storage usage. Not because they are actually storing anything extra, but just as a policy.
If you do this for a year, you are charged for storing 113MB on their service. You still only have 1K actually stored there.
Cloudflare still wins this estimate at 13c, and AWS remains expensive at 55k.
I’m assuming the bandwidth traffic helps improve Cloudflares other offerings (eg, insights).
> R2 will zero-rate infrequent storage operations under a threshold — currently planned to be in the single digit requests per second range.
If you a) don't bill for those in the 90th percentile of access and b) aren't too specific about the point at which you start billing, then you can eliminate a huge amount of complexity.
From there you could give users the link to R2 to get the cheaper bandwidth/usage fees.
Cloudflare also have additional services like Image Resizing which would completely remove 3rd party services like Imgix.
There's too many companies paying thousands for S3+Imgix which could easily save 80% of their bill with alternative solutions (haven't tried CF resizing yet)
My post about it https://pedrogomes.medium.com/how-to-save-90-of-your-imgix-m...
I've tried to migrate to cloudflare a couple of times.. I've never was able to find a way to setup routing for each `/path` of my website. I have different repos each controling one aspect of my website. This is a breaze to setup using cloudfront `origins`. My best guess is that I have to setup a Nginx between my content and cloudflare.
Maybe its me but I've always found cloudflare UI/UX confusing and hard to use. AWS is very bare metal UI, but it actually makes sense.
Hopefully as Cloudflare is launching new products, like R2, will force them to improve UI.
Problem remains using AWS Cloudfront I can route content to different sources.
I can't do this with cloudflare.
Anyone tried using User > Cloudflare > S3 directly?
Seems slower than User > Cloudflare > CloudFront > S3. (Using this for website CDN)
Less hops generally means better performance.
The other interesting thing is not all bandwidth is created equal. We provide services like 1.1.1.1 and infrastructure to root server operators in order to have a “currency” that’s helpful to ISPs who invite us into their networks. The number of times we’ve gotten invited into some telecom’s network because the network admin’s favorite sports team (or whatever) uses the free version of our service is… fascinating. And since every server in our network runs every service, once we’re in for one thing means everything we do in the region gets better and less expensive to operate.
This means, counter intuitively, as we add more locations to our network our costs generally go down, not up.
It always baffled me that regional ISPs weren't selling that last mile bandwidth to customers for real dollars. Even over DSL lines in the late 90s one could have run an amazing VoD service.
https://www.cloudflare.com/network/
But we deploy enough equipment to meet local demand (so will roughly correlate to population served). And that makes sense for ISPs to support because the alternative is them paying for back haul bandwidth themselves.
Given the rather large margins of home ISPs, it seems like they have you over a barrel if they set the price point sufficiently below the cost of transit without peering...
In a world with no limitations, we're free to no longer be creative.
$0.015/GB/month means $150/10TB/month, but you can buy an 10TB USB hard drive for around $150-250 which is likely to last at least 10 years on average, so the service costs on the order of 100 times the cost of just the drives.
Though maybe CloudFlare is not the best suited company for the "lots of cheap drives for backup" business.
- Electricity costs
- 99.999999999% (eleven 9’s) of annual durability
- Speed of your HDD (around 200 MB/s?) vs speed of R2
...
By the way this is the SLA terms from Cloudflare's website:
Q: What type of Service Level Agreement (SLA) do you offer? A: Cloudflare provides a SLA for our Business and Enterprise plans.
Cloudflare's Business plan offers a 100% uptime guarantee. In the event of downtime, customers receive a service credit against their monthly fee, in proportion to the respective disruption and affected customer ratio. You can view the full Business plan SLA, here: https://www.cloudflare.com/business-sla/
Cloudflare Enterprise customers receive a 10x credit (included in the Standard Success Offering) against the monthly fee, in proportion to the respective disruption and affected customer ratio. The Premium Success Offering includes 25x reimbursement uptime SLA. You can view the full Enterprise plan SLA, here: