Hacker News Reader

Hacker News

Comment by oolongCat | Hacker News Reader

Parent Full thread

oolongCat

What are some better alternatives to s3?

View on HN

Colocation hosting your own servers around the world... you know, the original way.

Is that really cheaper than S3, though?

Yes, way, way cheaper. Amazon egress bandwidth is 10-100x more expensive than IP transit.

Way cheaper. Amazon is a middleman for hardware. Even at smallest scales the cost of buying a server pays off within a year. In exchange you get convenience and the ability to scale up on demand without buying more hardware.

I'm sure buying the server pays off, but what about running and administering it?

What's the minimum commercially you'd spend to get cross-continent redundancy on hardware you manage in-house. Just the accounting cost of paying wages for technicians in two countries probably means you need to be a quite large company.

There are middle grounds, managed co-location say, but then you're not likely to save much over using AWS are you?

Even managed co-location is cheaper than AWS when you factor in the egress bandwidth charges at scale.

Another way to do it is to use dedicated servers. The datacenter will take care of repairs and remote hands functions and all you need is to manage the server from the OS level upwards. No need to have technicians in multiple countries, everything can be done remotely.

Yep this is what we do. It works well.

You still have to manage/ administor the aws instance. It only really saves you from the overhead involved with hardware failures.

Massively cheaper at scale. Do the basic math.

can you explain the math? I honestly don't have a working knoweldge of the inputs, but let's say i wanted the equivalent of ec2? How would I go about setting it up and the costs? If I wanted to serve a few million english speaking people my web application, how coud I calculate the price? Do I literally buy my own hardware, or I rent hardware at a dedicated rate? These are real questions btw, not rhetoric, it's hard to find a solid blog post or comparison of setting this up and the long term cost v convenience benefits of scaling this because almost everyone uses cloud providers, and by cloud providers I mean digital ocean, AWS, azure, rackspace or now google. Heroku (last I checked) sits on AWS as well and maybe even ngineyard does. actually pretty curious now if a few million people would be worth running your own gear, and if running that gear was lease v buy

The cost difference depends on how much your administration costs go up because you need to do things yourself, rather than through the AWS API. That's why it's hard to calculate.

Even at Amazon's cheapest posted bandwidth tier you're paying $50+ per terabyte for bandwidth.

Any dedicated hosting company (OVH, Hetzner, etc...) will cost less than 3% of that (and that's on the high end).

Bandwidth is cheap, but AWS marks it up roughly 30x-50x what you'd pay in a dedicated environment.

Then why isn't anyone running an AWS-like service focused for bandwidth-intensive applications hosted on "OVH, Hetzner, etc..."?

It depends on how you define AWS-like services. Lots of VPS providers cater to bandwidth-intensive applications and are hosted on cheap dedicated servers.

Many people do. S3 is convenient though. It handles durability, replication, uptime, etc... and even with the insane markup on bandwidth many startups / companies can afford a $10k/mo bandwidth bill because it's cheaper than hiring some devops and a team to build out the appropriate infrastructure.

That's what I meant with "really cheaper". Sure, bandwidth costs are lower with Hetzner, but that's not everything.

But you don't have to store and serve all the content from Amazon or your own platform. You can easily set up aggressive caching, in a network you control, backed by S3.

And you pay premium for premium bandwidth. You can't just assume that all your users will have a good experience if you buy hardware and bandwidth from the cheapest providers around.

Amazon bandwidth isn't particularly premium. It's just expensive.

Depends on what you compare it you. Amazon sells you not only the hardware, but also the management services.

Even cheaper and easier is just to rent dedicated servers or VPSes.

One of the earlier Imgur style hosts, ImageVenue, did this very successfully (they were extremely large in terms of traffic at their peak and for the time, until better options came along).

The founder, Vlad, used to rent out cheap dedicated boxes with about 500gb of bandwidth per box for $60 to $80 per month. He'd cycle upload machines - eg 123.imagevenue.com - and fill up the drives on them until no more space was available; when bandwidth was exhausted, that was that for the month. Typically image file downloads would decline on any given upload over time, so machine resources would free back up in terms of bandwidth and deleting images that weren't viewed after N amount of time.

For image-heavy use I found its just cheaper to build your own box and cohost instead of renting.

How do you figure that? You can rent the exact same server as a dedicated server. The difference is, you have no capital outlay up front, you don't need to ship and rack it or replace broken parts and there is only a month-to-month commitment.

The boxes with lots of disk and memory are much more expensive when rented vs buying. SSDs are super expensive to rent, for example.

I made a Skylake Xeon system with all enterprise components for $2k. You can't even rent Skylake Xeons anywhere right now.

I've never had problems ordering custom hardware from my dedicated server providers. At worst, all I've had to do is pay a one-time fee for a particular component. SSDs have never been a problem. Neither has any Intel CPU.

It of course depends on your provider.

Rent vs buy depends a lot on the circumstances. Maybe if your requirements don't change and you don't upgrade your server hardware it's cheaper in the long run to buy. However, when I've kept a server for more than a year, I've always been able to negotiate a discount.

You didn't post your config, so I can't tell how exotic your requirements are.

s3 is awesome. Keep using it but put your own boxes in a colo with a direct connect to aws (not all colos offer this unfortunately). Your boxes can then g3 20+tb of ssd cache in front of your raw pb+ datasets in s3.

https://aws.amazon.com/directconnect/pricing/

Not sure if this is just for your private photos, but it's cheaper: https://www.backblaze.com/b2/cloud-storage.html

Use S3 but put a cache in front of it, at the very least.

Reddit use Cloudflare on both reddit.com and redditstatic.com

Reply on news.ycombinator.com