Hacker News Reader

That's exactly why I started Ploid.

A consulting customer came to me a year ago, with a growth from 200TB/year in data production to over 6PB/year and their budget couldn't sustain that jump (or anywhere close to it)

Having come from the mass-facilities and data center space with MagicJack, I knew the wholesale cost of bandwidth, power and drives were continuously falling.

There are certain clients and use cases that need access to their data all of the time and the very bones they are built on is based on collaboration (Genomics).

For example, this client is now storing 6PB of data with us, 3 copies in separate data centers. We are half the price of S3, and we include all the bandwidth for free, but limited to a 10GigE per PB stored. This has worked out extremely well - we were about 20% (!!!!) the price of Amazon after you factor in bandwidth.

There are lots of challenges we faced, like over zealous neighbors in the environment, storing lots of small objects and high usage of ancillary features like metadata but for customers of any size. By putting the "tax" on bandwidth, a lot of these business cases are solved. I see why Amazon does that.

AWS is truly great, but as you get into very high scale (specifically in storage - 2PB+), it becomes extremely cost prohibitive.

"By putting the "tax" on bandwidth, a lot of these business cases are solved. I see why Amazon does that."

However, S3 has the same egress pricing as EC2. Do you think it's really a "business case tax" they're applying across all services?

It makes a lot of sense to be able to run loss making products. Otherwise everyone would use S3 together with Google compute engine and Azure databases (let's assume they'd be cheapest). In this scenario all providers would lose out.

In the current world, they can keep prices for some products below costs but make their money with bandwidth and the other services people are forced to use to avoid egress traffic.

"In the current world, they can keep prices for some products below costs but make their money with bandwidth and the other services people are forced to use to avoid egress traffic."

Which AWS products are loss leaders?

S3 storage pricing is not exactly cheap. Neither is EC2 instance pricing.

"Otherwise everyone would use S3 together with Google compute engine and Azure databases (let's assume they'd be cheapest). In this scenario all providers would lose out."

No, S3 would do well, GCE would do well, Azure would do well. Providers only lose out to the extent their products no longer compete on merit alone.

I can imagine that this is a good reason. Otherwise they could make bandwidth cheaper so that people who cannot move everything can at least move part of their applications.

I think the three providers are smart enough to know why they charge that much for bandwidth. And this is the only reason I could think of why all 3 of them charge that much. And I'm pretty sure that some products run at a loss, they do for nearly every company. But AWS won't tell us which ones.

It's reasonable to think that S3 is loss making or about breakeven on its own but recoups costs due to bandwidth charges.

There's still latency, you know ;).

I guess the latency between AWS Frankfurt and GC Belgium should be low enough (5-10ms) to use it for most applications. E.g. storing large amount of data at one provider and renting compute engines for processing at the other one. The latency shouldn't be an issue there, as long as the throughput is high enough.

Can confirm on this, storage for a lot of stuff is in S3 and compute is GCP preemtpibles. Works if you have a small dataset which requires a large volume of compute.

Is that cheaper than using Google for storage as well? Or are there other reasons for that setup?

Bit of both, no point moving it as the automation/clients that dump data to S3 make it quite hard to change.

GCS supports the S3 API modulo resumable uploads (we do them differently): https://cloud.google.com/storage/docs/migrating

Feel free to send me a note, my contact info is in my profile (I helped build preemptible VMs and I'm sort of fascinated you're doing this).

Yes - I think it directly applies to EC2 as well. Still an underlying commodity

Could you link to ploid please? Neither google nor bing can find it from that name.

http://www.ploid.io - found at: https://www.linkedin.com/in/brandonkruse

Dead? I get either site not found or a domain parking page.

Maybe it's under a different domain name. I was hoping to draw @bkruse out to tell us where we can sign up for Ploid because I'm interested too.

(repost)

Sorry for the delay - yes it's ploid.io - nothing up there yet. We've been in stealth mode while we've been building the system for our first client - HudsonAlpha institute for biotechnology Feel free to ping me at brandon at ploid.io - happy to share any insight we've gained!

Yes, you've tickled my curiosity too but I can't find you.

Sorry for the delay - yes it's ploid.io - nothing up there yet. We've been in stealth mode while we've been building the system for our first client - HudsonAlpha institute for biotechnology

Feel free to ping me at brandon at ploid.io - happy to share any insight we've gained!

> The bandwidth costs are so far out of line with what the network transfer actually costs, it just feels like price fixing between the major cloud players that nobody is drastically reducing those prices, only storage prices.

Yes, it makes me wonder. For many applications, the ridiculous bandwidth costs will be substantially higher than compute costs, so it makes no sense that nobody in this supposedly competitive market wants to compete in this area.

> There are several providers that offer a unmetered 1gbps line PLUS a dedicated server for ~600-750/mo. Providers like OVH offer the bandwidth for as little as 100/month. ( https://www.ovh.com/us/dedicated-servers/bandwidth-upgrade.x.... ) I am just not sure how amazon can justify a 160x price increase over OVH or a 30x increase over dedicated server + transfer.

OVH and other large volume providers are probably relying on the fact that many customers won't use 100% of their purchased capacity, but even taking that into consideration, AWS/GCE/Azure bandwidth pricing is insane.

It is indeed curiously unspoken among the major providers. (Softlayer too raised all their bw rates when IBM bought them.)

Part of building a moat? On one side we have a big stack of custom APIs. On the other -- once you have a few hundred terabytes of data in our systems -- it will be very costly to emigrate.

Exactly!

I knew as soon as I saw the headline that the would be reducing storage costs (who cares????) and not reducing the bandwidth costs which have been stuck for years, which are price-fixed with the other providers, and which account for the vast majority of our S3 bill.

Are bandwidth in costs free? Are you referring to bandwidth out costs? I don't work for them but I imagine they model bandwidth out as disk reads for accessing and reading your data. Think about how much it would cost to read at the rate of a 1 gigabit line from disk continuously for a month. Put a CDN in front of S3 if you have high bandwidth costs and low diversity of files.

Due to the high traffic volume, they peer with most large providers. Of course they need to pay rent on the fiber that they rent between DCs and peering points, but at this level the cost should be rather low. And a lot of the traffic will be free, especially in Europe where you can peer with many providers on exchanges.

That would go a way towards understanding the over-inflated egress (bandwidth out) costs, if S3 did not already charge for disk reads.

S3 already have request pricing in place: https://aws.amazon.com/s3/pricing/#Request_Pricing

In addition to that, the pricing for "Data Transfer OUT From Amazon S3 To Internet" is exactly the same as for "Data Transfer OUT From Amazon EC2 To Internet", so this is not specific to S3 but to EC2 and AWS as a whole it seems.

AWS appear to have really expensive egress costs (or really profitable egress margins) compared to OVH and Hetzner. If so, then something is stuck, either the costs are not being addressed or the margins are not being passed on.

CloudFront costs are super expensive outside of the US and Europe. We are a small company and just in south America we are paying 1000USD per month for 4 TB of traffic (0.25$ per GB). Based on traffic alone we are loosing money with some customers.

Why not use something like CloudFlare for ~$200/mo?

Cloudflare doesn't support large media files if I recall correctly.

Also, it means using DNS with them as well.

Seems like you should use CloudFront for things that are both small and need to be fast (HTML/ JS/ CSS), and then use a separate service for hosting your fat media files. Heck, rent a pair of servers with unlimited bandwith somewhere for $100 a month. Yes the bandwidth may be a bit oversold and spikey, but if a giant file download slows from 10 MB/ sec to 5 MB/sec for a bit, I doubt it matters THAT much.

Check out Fastly or CacheFly.

We have tried CloudFlare around 2 years ago, we had complains about performance issues so we changed back to CloudFront.

Here is a blog post that shows the same problem we were having: http://goldfirestudios.com/blog/135/CDN-Benchmarks%3A-CloudF...

For many hosts, bandwidth is what they are really selling. You may rent a server from them and they may make some money from that over the capital costs, but the amount they are charging for bandwidth in comparison to what they pay for it is where the real money is made.

Also from OVH, have a look at OVH Object Storage, which is pretty much S3 but for 1/10 of the price [1].

There is also a GitHub comparison for all object storage providers (this price reduction isn't included yet as I see) [2].

[1] https://www.ovh.com/us/cloud/storage/object-storage.xml [2] http://gaul.org/object-store-comparison/

I just tried to sign up for an Object Storage account. The form assumed I was from Canada so I couldn't fill it in.

I use s3 as a webserver for static pages. Total bandwidth costs: 17 cents a month. 14 or so cents now. And I have some tens of thousands of visits per month. The only cognitive thought I've had to put in since setup was questioning one aspect of a bill (it amounted to pennies, but I couldn't figure it out, and maybe they were overbilling bigger entities). And they gave me a month free (including DNS costs).

I wish the Route53 would go down... Hard to beat, even if I were running job-replacing amounts of income.

I assume from these numbers that you only have a few GB of traffic per month. If your page ever became popular this would quickly change.

Or he has a very skinny page :) Not a 3 MB js monstrosity.

Even with 100kb (assuming it's not only text), 1mln pageviews per month across all sites will bring you to 100GB, a point where a small vps at DO/Vultr/Hetzner/OVH would be cheaper than AWS. And since he was talking of several pages, 1mln pageviews isn't even a very popular page.

If you are getting 1m pageviews, you should be able to make what, $5000 in adsense revenue?

At that point, I think the reliability and multicast nature of Cloudfront or some other CDN is worth the $5 premium over DO/Vultr/OVH....

A) I think the revenue would be far less than that

B) Not everything is about selling ad space, or even generating revenue.

I think it is possible to draw some conclusions from the volume discounts AWS is giving on data transfer. From 10TB/month to 150TB/month the price drops $0.09 => $0.05. I don't see how this kind of change in individual customer's volumes would make any difference on how much Amazon pays for the traffic. So I assume it is quite profitable for them to be selling at $0.05 (and lower) and the higher prices are just because smaller customers accept them.

The bandwidth seems to be used for trapping users in. You don't get charged when you're uploading data to AWS but downloading it back is quite expensive.

I wholeheartedly agree. The bandwidth cost reduction is long overdue.