Hacker News Reader

Comment by iskander | Hacker News Reader

I want to store a few dozen TB of genomics data in a publicly accessible way. Are there any better alternatives to S3 or Google Cloud Storage? I've been waiting for Cloudflare's R2 but the beta is not yet open yet and I'm not even sure it would work well for me.

View on HN

Better in terms of what? There are many variables to consider, and some might be more important than others in your case. If you "cost" is the top priority, get a dedicated instance with unmetered connection and a 1 or 10gbps port, then you'll have a static price/month that won't surprise you. If latency/bandwidth is more important, throw a CDN in front of that instance, but price will vary more then as you'll pay per data served (in most cases).

You could use requested pays with s3 or GCP, which requires the reader to use their own project and pay for requests.

Spin up a GSuite Enterprise account and drop it all in Drive. Unlimited storage in Drive for enterprise accounts.

They really don't cut you off if you put 10TB of FASTQs in there and let strangers download them?

Wasabi [1] is admittedly up to 80% cheaper than S3. But it forces you to keep your files for 90 days at least. I saw it recommended several times as a cheaper alternative to S3-backed storage for OwnCloud.

[1] https://wasabi.com/

Wasabi's policy prevents reading more than you write. They're designed for backup, not publication.

If you just want to beat the big providers, use B2.

If you expect to use many terabytes of bandwidth, then it's time to rent a server or VPS and probably set up a torrent too.

How about B2 combined with Cloudflare? Egress from B2 to Cloudflare is free and egress from Cloudflare to the internet is free for their free plan, and perhaps competitive on their not free plan?

Maybe! You'd need to use workers for giant blobs of data, not their normal website plans, but those don't cost a lot.

It's probably worth setting up a cheap torrent-seeding server too.

There's always ftp.

Reply on news.ycombinator.com