Ask HN: Have You Migrated from AWS S3 to Cloudflare R2? | Hacker News Reader

Ask HN: Have You Migrated from AWS S3 to Cloudflare R2?

52 pointsezekg25 comments

If so, how did you do it? How did it go? How are the savings? I have TBs of data I need to progressively replicate to R2. The original blog post on R2 talked about an automatic migration tool for S3 to R2, but I find no mention of it anywhere outside of that blog post.

View on HN

It’ll cost you $90 per terabyte to exit from S3.

If I’m reading this right, the first 100GB per month out of AWS is now free?

https://aws.amazon.com/about-aws/whats-new/2021/11/aws-price...

So you can subtract $9.

Only if you are small enough to be paying full price, in which case how do you have that much data?

Can you explain what you mean?

How much do you think companies pay for data egress from AWS?

Can't disclose specifics but if you look around on internet (AWS EDP) you can see organizations that have a certain level of spend have massive discount on egress/S3 storage pricing. Discount on top of the tiered pricing.

e.g. US Ohio: Greater than 150 TB / Month $0.05 per GB, then another XX% on top of that.

Depending on your monthly spend, you can get private pricing agreements for most AWS services. But I can't really get into numbers because I like being employed.

Terabytes of a data is small. You’re not getting discounts from Amazon at that scale.

For this reason, we now switch between sources in s3 and r2. Any artifact stored after 1666666666666 (25 Oct 2022) is in r2 for us (yeah, we universally include timestamp in s3/r2 keys/prefixes).

Do you intend to migrate to backfill r2 and make it primary/sole object store?

Not planning to move all of it (in fact, doing so would be costlier given our current workload patterns which are time sensitive and hence over a period of next few months, S3 usage should dwindle organically), but rather manage the data out with lifecycle rules eventually into S3-InfrequentAccess.

Came to the same conclusion. Will probably do this so that we eventually migrate to R2, without the need to backfill.

R2's compatibility with the S3 API still has a ways to go. I tried to port over a relatively trivial app written on the AWS SDK and found:

* Request header signing is not supported (ok, fine)

* Tagging is not supported (showstopper)

How often do you use bandwidth use from those TB? IIRC, s3 can be cheaper b/c of the tiered service options, but if you are eating in the bandwidth bill from amazon, you may see joy from R2.

Beware: the S3 tiered options also have additional fees, like minimum object sizes and minimum object lifetimes. For example, in a situation where a file is getting frequently overwritten, it will cost more to store it in one of the "lower priced" tiers than in regular S3.

Intelligent tiering doesn’t have this multi-write issue. For the most part it’s a guaranteed win over standard tier for objects above 128k.

You can implement compression in an AWS Lambda function. You might save some money by compressing your data before egress. I don't know if R2 has a similar unzip function.

probably with a worker you can pull that off

We migrated from S3 to Minio, an S3-compatible storage. However, we mostly store backups (6-8 TB) there so we don't care about five 9s or performance that much (if we get 80% out of it, it is good enough).

Pricing is far better with that amount of data if you self-host.

> We migrated from S3 to Minio

Hosted/SaaS Minio or self hosted?

Self-hosted

What kind of project would you use to self-host?

We use colocation, because it's the best $/GB one can get. Maintenance overhead is small, a couple of hours per month.

We moved from Minio to Wasabi. Big improvement in costs.

Meaning, you were paying minio, now using wasabi? You weren't self hosting, right? If you were self hosting, I'm curious how wasabi is cheaper than minio.

The costs per TB for self hosting is more expensive as what I pay now at Wasabi. Take in account (redundant) disks and power

Minio SaaS?

Reply on news.ycombinator.com