Hacker News Reader

Comment by marcc | Hacker News Reader

IMO, the best feature Cloudflare added to R2 was the automatic migration. This makes sense for a CDN to offer, but if R2 works, this makes it pretty simple to migrate.

"After specifying an existing storage bucket, R2 will serve requests for objects from the existing bucket,"

I've Beene evaluating R2, but this migration path makes it dead simple to use. I just point my code to read/write from/to R2 instead of S3, and I'll get egress fees from AWS during the migration, but then that's it?

View on HN

> I just point my code to read/write from/to R2 instead of S3

This is probably worse for CloudFront than for S3 itself.

Double paying for storage is still going to be cheaper than egress fees overall.

Good point about double paying for storage. There's a long migration problem here to clean up. Most of my buckets have some frequently and some very infrequently accessed objects. To address this, there needs to be some sort of an active migration tool? Does R2 have this built in?

I’m not sure exactly what you mean by active migration tool, but my understanding is that they copy objects from s3 to r2 on the first request. So if you have infrequently accessed objects they won’t be copied and you won’t be double paying storage until it’s first requested.

Just an idea, maybe the first fetch from S3 should be allowed by R2 do delete the original object from S3 too, so that eventually we're only left with two mutually exclusive sets of files (and no double storage)

I thought about that too. It's could be a good solution because the challenge otherwise is going to be listing all objects in the bucket and comparing to what's in the R2 bucket, right?

Based on what I’ve seen in this convo:

- Set up the S3 mirror into R2

- Migrate your code to read from R2.

- Set up SQS to populate with S3 Create events. SQS listeners just make a GET request to R2 for that file.

- Generate S3 events and populate SQS by running List operations or by abusing S3 Lifecycle Management.

- Let it process.

- Switch writes to R2.

This all assumes you can’t delete from S3 till R2 is fully ready. Depending on the application, you could switch over writes to R2 in a different step and also possibly delete the S3 file in the SQS processor.

I like Cloudflare for its CDN, but considering its competitors and migration effort I think that only time will tell.

hard same, the automatic migration is a big deal!

insert doubt

Why would you doubt it? Cloudfare is a legit company and a migration workflow like this does not sound like a feat of engineering to accomplish

What part is OP missing?

Maybe the point is that just having the S3 and CDN isn’t enough to move customers.

If you’re using AWS, S3 (and Cloudfront) it will just be a tiny part of your infrastructure, why would you move that out of AWS and lose the integration coming from having everything in AWS?

Because it's significantly cheaper and diversifies your presence across multiple providers.

It's a good idea to split origin and CDN, this way requests can be served from cache while origin is down or overloaded.

Plus I'd challenge the suggestions that S3 and CDN are a tiny portion of the infrastructure. For a lot of sites it's a significant chunk, things remaining wouldn't be many, maybe compute, DNS, and CI.

> diversifies your presence across multiple providers.

In this case, for most AWS/CF customers, that diversification is of the "downtime on either represents downtime for their customer" variety. (I'm still a fan of this offering and the pressure it put on the current high price of egress bandwidth, of course.)

If S3 was down would R2 be down for the customer?

If R2 was down could you re-route back to S3?

Diversifying infrastructure provides additional options to mitigate outages.

If S3 was down, R2 would be up but the rest of your app on AWS would probably be down in the affected (based on the cascading failures in prior S3 outages) so most AWS users are going to be down.

If R2 was down and you didn’t rely on anything else that relied on R2, sure, you could repoint DNS and/or issue S3-direct URLs and then clean it up on the way back out.

“Additional options” doesn’t come without “additional complexity” and that complexity has a cost.

R2’s a great cost-saving move here; I don’t think it’s a reliability improvement generally.

It might be only part of your infrastructure but on many projects it’s a significant fraction of your total expenditure and storage egress is one of the things limiting use of external services. Having a cheap data access option makes it easy to reconsider those choices when you don’t have to factor egress into the cost of trying an alternative.

> Double paying for storage is still going to be cheaper than egress fees overall.

because of this?

There are many applications that have no dependency on AWS at all, other than S3. This is one of the reasons projects like Minio exist.

If you don't care and just need some blob storage, this could be great.

Zero egress costs for assets is a very strong reason for people to move. For public assets, it's a complete no-brainer.

I kinda think people will use the S3 proxy feature of R2 for that, which doesn’t really move customers over.

Then why not store it on Cloudflare itself and save on half the storage, plus less latency?

Makes zero sense to keep public assets on S3 in this case. Only when you need ACL integration or glacier would S3 make sense.

Reply on news.ycombinator.com