Hacker News Reader

Comment by jorams | Hacker News Reader

I can give you the reason we moved from Digital Ocean to Google Cloud: Spaces (object storage) was ridiculously slow and unreliable. There were several incidents per week where Spaces had bad connectivity or simply seemed to have crashed entirely, and that's not something we can deal with for production traffic. We now run a cheap CDN in front of it to cut down on the ridiculous bandwidth costs, but Google Cloud Storage has been very reliable and fast.

View on HN

This. There is no alternative for high performance object storage outside of AWS S3 and Google Cloud Storage. Even Azure's offering is wonky.

Lots of providers claim to offer object storage, but try hitting them from couple thousand cores and they all tend to immediately fall over.

That's not a very common use case though. Most companies don't need to DDOS their cloud provider.

25Gbps is 250 cable Internet users downloading the release of your new software. What you call a DDOS is actually a woefully underwhelming instantaneous transfer rate.

You should put your objects storage behind a CDN if you expect that many downloads. That's much cheaper and faster.

If you're distributing releases (or anything, really) users should not have to go directly to your object store...

AWS S3 buckets have a limit of 5gb/s access to a single bucket.

Unless that limitation has changed in the last couple of years; I can easily make a system beat that, if that's the requirement.. and ultimately it does come down to understanding requirements. :\

I think generally people forget that cloud is just computers too, it's really not anything special, and amazon/google/microsoft are solving the general case (and, doing so well, actually) but this comes at a high premium.

It's 25Gbps now[1]. I have a rule of thumb: if I haven't refreshed something I think I know about AWS for more than half a year, it's probably out of date.

[1]: https://aws.amazon.com/premiumsupport/knowledge-center/s3-ma...

That the throughput available to a single EC2 instance. S3 as a whole will do hundreds of Gbps no problem.

There's options outside SaaS object storage though, eg running your own MinIO or Ceph, + DAS (or SAN) at your colo. Yes it's some hassle but if you have significant expenses from SaaS it might be worth checking out.

I run my own Docker registry for my personal projects backed by DO Spaces. It was so slow that I couldn't actually believe it, and had to instrument the registry code extensively to account for all the time spent in the storage API, to see what the actual problem was. Turns out Spaces is really, really slow.

is it that bad? we are thinking about bootstrapping a project, which will use spaces for file storage and we will be using them frequently.

I tried it for a small personal project, just for somewhere to put some small images. On pages of about 30 images, usually one of them just wouldn't load. It didn't look reliable to me either.

Provisioning of new Spaces was recently disabled in some regions while they add capacity. Vaguely concerning IMO and has caused me to seriously reconsider using DO Spaces for a new product I'm building.

https://www.digitalocean.com/docs/release-notes/upcoming/spa...

I do know that they have limited the creation and looking at the docs for limitation, it seems like they have a 750 requests per second limit per space, which might not scale well. My idea was to create multiple spaces to distribute a load, e.g. sharding per customer, but if the availability is not that good, I might have to re-consider our choice.

The main reason we wanted to work with DO is that the egress traffic cost is reasonable, unlike AWS or Google Cloud where it is questionably high.

It was that bad 2 years ago, in the AMS3 region. I don't know how it is these days, but I expect after 2 years it has matured at least a little bit. They also now include a CDN, which might improve things a bit if it reduces the load on the buckets behind it.

Honestly it soured my opinion on DO a bit. I'd previously had a droplet there for several years, which never seemed to experience any issues at all.

In our case, we can not use the CDN, unfortunately, because the objects change a lot.

Reply on news.ycombinator.com