Amazon S3 Pricing Changes Effective April 1, 2014 | Hacker News Reader

Amazon S3 Pricing Changes Effective April 1, 2014

aws.amazon.com

344 pointstluthra181 comments

My concern is what it always been for AWS, no first party support for limiting my bill. The best effort from AWS is just an email notification when they "think" my bill is over a limit. If I can't limit my risk to a surprise bill of thousands of dollars after missing a midnight email, I still can't use their service at any usage based price other than free.

This is very hard for them to do.

For example, what if you have set up a bunch of auto-scaling instances to handle heavy load. During the middle of the night, your application suddenly goes viral in Indonesia or Brazil or some other large country in a different timezone from yours.

Your bill goes over its threshold - what does Amazon do? Do they terminate some of your instances (potentially losing production data!)?

The problem is that the exact times that "intelligent" bill-limiting features are likely to be used are the exact instances in which they could have absolutely disastrous impact if they make a mistake[0]

I haven't experienced this myself, but I've heard Amazon is actually rather lenient when it comes to waiving fees for clear mistakes (accidentally provisioning an expensive instance that you didn't mean to) - their customer support is supposed to be very good, though I have not had to use this myself.

[0] By "mistake", I mean that they are following your instructions correctly, but your instructions happen to be incorrect for handling this situation.

Yea any intelligent bill limiting needs to be done with context into what each instance is doing. It's possible for them to implement something that can handle that, using tags and such, but I would rather they spent that time improving reliability infrastructure.

It's not terribly difficult to implement one's own bill limiting system using the API.

It seems impossible for a single solution here. One business would rather see instances go down, another might need ledger logs no matter what and would rather turn down customers at the door, etc.,

The right way to do it (currently) is to model the costs yourself and tie it in with your operational automation, then determine what your best course of action would be during a shitstorm. Generally, I would wager the best option is to replace your front page with a static html "overwhelming response" page, maybe with some light vector graphics.

> Do they terminate some of your instances (potentially losing production data!)?

If you have opted in for the "don't let me spend more than $x," then yes.

I'd say the more intelligent way to head any of this off is to create billing alerts which you can now do and set it to page / email / post to an endpoint when you reach $X or X% of whatever worries you.

Create tiers of things that email / SMS additional people as the numbers go up if you want to be very careful etc...

I'd second that! It's not complicated, especially for smaller entities. "I'll use your services, but don't go over ths amount; without authorization." If my wiz, bang app, website has that much potential--I'll get the money to you. I don't want to wake up to a big bill. To be honest, if they were very clear about pricing, and stop/loss orders; I think more people would sign up. It's about honesty, and a minimizing the variables. Oh, and fire any MBA advisors; they think they are slick, but your average dude us starting to look at the fine print, and has become very sensitive to being taken advantage of. By the way, off topic, but relevant. I looked into esurance. Very good price. Then I realized the premium was only fir six months. I always wondered how they could afford all those rediculous commercials.

6 months is industry-standard quoting for auto insurance -- not sure why, but it is. What product are you talking about?

You can set up billing alerts which sends you alerts if your bill exceeds a value (set by you)

While not the best solution, you can call Amazon, speak to someone to have your "surprise" bill mitigated.

This is 1st hand information from a AWS Rep I once dated.

That is so wrong it's not funny. Call? Isn't our use of computers predicated on the notion that we don't want to do manual labor?

Moving your mouth muscles isn't significantly more manual then moving the finger muscles you used to type this comment.

In either case, expending a few neural cycles prior to the action would be beneficial.

My main gig revolves around bill auditing. You'd be amazed at the number of Fortune 500's that are supposedly driven by technology that have very manual processes around billing.

I work at one. I know, sadly.

I think thought that it's a failure of technologists to properly address the challenges of billing.

I used to do a billing system that processed about $15 million a year. To my horror, I realized when I joined that all chargeback processing involved someone in accounting going through the huge envelopes of chargeback claims + photocopied statements that they received in order to determine what to challenge. The online payment processor they used at the time didn't have any option for presenting the documentation online....

Well, you can do a ticket, too.

Please, point out to me the system that adequately resolves unusual billing situations without human intervention on both sides. I'd like to know what has set your bar so high that it prompted this level of entitled commentary.

Personally, I find it hilarious that a simple verbal interaction with another human being is now categorized as "manual labor".

Sat next to a businessperson on a flight to Vegas, and he was opining that "schmoozing" is becoming a lost art.

I'm all for being able to initiate billing inquiries online--in fact, I'm ditching Bank of America because I can't dispute charges on line.

But in Amazon's case, this isn't a dispute, rather this is asking them to forgive a (hopefully) one time transgression, which is much easier to do by speaking to someone, than filling out a form.

I find it similar to haggling for a temporary reprieve in my fantastically high cable bill.

Small yes. Large bill wasn't so easy. Once I had an excess $40 added to my bill after a network attack. I was able to down to $10 after a few emails back and forth. Another time I had something low like $10 and they credit me right away without problem. Depends on how large your bill...

You live in a very different world from me if a "large bill" is an extra $40. For even a small company like who I work for, any fluctuations in our AWS bill under $100 would probably go unnoticed. Our total infrastructure cost (of which AWS is a small part) is in 5 digits.

Personal website of course it would be a big sum...

Surely your dev team laptops total 5 digits already.

According to her, she would routinely have one-time massive fee's credited ($1000+).

I'd assume since it's a spike in costs, Amazon has more interest in keeping you as a customer, at your much lower typical usage, than to have you abandon, even if you have a huge bill.

Obviously if this was a typical problem, you would need to rethink your architecture and infrastructure needs.

You actually can do this yourself quite readily, through use of CloudWatch billing metrics (which can be broken down by individual service). Define some alarms based on billing thresholds, and perform automated scaling actions in response.

AWS can't exactly do this for you, as explained by previous commenters, but they do give you the basic features you'd need for implementing it.

If you can get something to monitor your billing rate, you might be able to get PagerDuty to call you if your billing rate goes over x per hour.

There is a newrelic plugin that does this. It can graph your costs and raise alerts.

AWS Costs & Usage Analytics http://www.newvem.com/new-relic/

If you're able to set up a service that risks a high bill, but cannot set up something that turns an email into some other action (anything from a text message to automatically shutting down your service), I don't think the limitations of AWS are the problem...

(Having said that, AWS could really do a lot better with it's notification targets than just email.)

Any service risks a high bill. Any object hosted on S3 could be the target of a DDoS attack, for example.

Hence the reason why you should be signing your S3 URL's.

I accidentally setup EC2's in another Region (and effectively "forgot" about them. Woke up with a 1.2k invoice + $250 more in charges for March.

Amazon voided all charges without question. Their level of customer support is off the charts. But AWS would never be my first pick to start up something. Your costs will be way way lower when going dedicated. Obviously you should be able to scale, but I think startups generally overestimate their success when it comes to that, and wasting a lot of money for things they have no use for yet.

You could have your AWS deployment strategies ready for when the need would arise..

EC2 pricing is more in line with other vendors. S3 is silly cheap.

Amazon also has a startup program where you can get like $7k or similar free AWS credit when you register. They do some basic checks, but it's pretty easy to get IIRC.

Key quote: "We are lowering S3 storage prices by 36% to 65%, effective April 1st, 2014."

http://aws.amazon.com/s3/pricing/

Unless you're storing massive amounts of data, no point in using Glacier now, as it's still $0.01/GB (vs $0.03/GB for standard durability).

And wow. Hell of a price drop.

I don't think a few TB counts as "massive", and that still saves you over a grand a year. Why move backups to a 3x more expensive service, what's the upside?

Not having to wait 4-8 hours to access them. Also, pulling from S3 costs you nothing (if you're in EC2 or other AWS services). Glacier access costs can get expensive depending on the quantity of data you need to access in a short period of time:

"Glacier is designed with the expectation that retrievals are infrequent and unusual, and data will be stored for extended periods of time. You can retrieve up to 5% of your average monthly storage (pro-rated daily) for free each month. If you choose to retrieve more than this amount of data in a month, you are charged a retrieval fee starting at $0.01 per gigabyte. Learn more. In addition, there is a pro-rated charge of $0.03 per gigabyte for items deleted prior to 90 days."

http://aws.amazon.com/glacier/pricing/

To store 1TB for a year in S3 is now $368.64

To store 1TB in Glacier for a year is $122.88

We're quibbling over ~$250/year?

What if you have 100 TB?

I would quibble over 25k a year..

>Not having to wait 4-8 hours to access them

I can't download the data that fast anyways, it is a total non issue for us.

>Glacier access costs can get expensive depending on the quantity of data you need to access in a short period of time

Yeah, it is for backups. That's why we are talking about how we use it for backups.

>We're quibbling over ~$250/year?

Times the "few TB" we are talking about, that is over a$1k a year, as I said. We're not quibbling. I asked a simple question. Why would I take the time and effort to move my backups from a 1c/GB service to a 3c/GB service? There is no upside. "You can pay an extra $1000 for no reason, which is a great idea because $1000 isn't very much" is not very compelling.

Glacier's pricing is actually very difficult to calculate. If you only ever store data (and never retrieve it), it's easy, but the moment you need to start accessing it, the pricing is actually very difficult to calculate correctly:

> Unlike every other service Amazon provides, with Glacier you're not paying for your net usage throughout each month: Instead, you're paying for your peak hour — and once that peak hour has come and gone, you're still paying for that peak until the end of the month, no matter how low your usage might fall. [0]

Note: I still use Glacier for long-term backup, because that's the exact use case it's meant for. But it's important to be aware of that cost: if I ever need to retrieve that data, I know I need to do it very slowly (and carefully) to avoid racking up a massive bill.

[0] http://www.daemonology.net/blog/2012-09-04-thoughts-on-glaci...

There's a few calculators that make it quite simple to figure out restore costs.

It's true that Glacier is a bit tricky, but you can easily cap your retrieval costs (with a bit of margin) if your use-case allows you to choose a maximum retrieval rate, rather than thinking in terms of retrieval amount. If your peak hourly retrieval rate for a month is N Mbps, your cost is no more than $3N. So for example, $30 for a month that includes 10 Mbps retrievals, $300 for 100 Mbps retrievals, etc.

A bit more detail in an older comment of mine: https://news.ycombinator.com/item?id=7147325

300% is still 300%.

It's only 300% for the first TB in S3. It gets progressively cheaper after that.

It would be interesting to go back through the Everpix post mortem docs[1] and see if a big price swing like this could have put them in the black (or close to it).

[1] https://github.com/everpix/Everpix-Intelligence

I had the same thought. Their last Amazon bill was $35k. I don't think saving $10-20k/month would have made a significant difference. They had many freemium users, and were not adding customers at a sufficiently high growth rate over the life of the company. They were trying to raise $5m.

The importance of this depends on unit economics. For businesses with historically tight margins (e.g. retail), small swings in input prices (labor, cost of goods) can make or break profitability.

Whereas software, if you're doing it right, should have much higher margins.

Sidenote: I'd be very curious to know what the "typical SaaS business" has as its gross margin, factoring in all sales against "cost of goods": servers, bandwidth, operations, etc.

80% seems to be the common 'educated guess' number:

http://www.quora.com/What-should-be-the-typical-gross-margin...

http://www.forentrepreneurs.com/saas-metrics-2-definitions/

http://www.forbes.com/sites/jjcolao/2012/10/02/running-a-saa...

It might have been enough for them to keep investor interest [new round of funding] and continue. It wouldn't have put them in the black.

https://github.com/everpix/Everpix-Intelligence/blob/master/...

From that, it looks like their loss over 26 months was ~$300,000. Their total AWS costs was less than $400,000 over the same period.

Downloading 1 GB = 4x the cost of the S3 storage at the reduced prices. If everything is viewed once [doubtful], that is 4x the price of the S3 storage. Given that ti would be impossible to reduce the total cost by even 50%, let alone the 75% they'd need to break approximately even.

I didn't see S3 broken out specifically anywhere which is why I'm just making an educated guess. :/

very cool repository, found the S3 data here:

https://github.com/everpix/Everpix-Intelligence/blob/master/...

So basically at their peak, they had ~174TB S3 and ~47TB S3 RRS. That month cost them ~$16.2k, whereas after April 1 they could have put everything in S3 for ~$6.7k. That's a big difference, but they were still getting taken to the cleaners over some other services like RDS.

Cool. So they had roughly (with rounding):

RRS: 2+3+6+8+9+18+19+22+27+34+42+48+7=~245 TB/months=~

Normal: 7+9+13+19+24+46+56+65+76+95+123+153+174+25=~885 TB/months

Difference:

$0.085-$0.0300=$0.055

$0.068-$0.0240=$0.044

End:

0.055 * 885 * 1024=$49,843 0.044 * 245 * 1024=$11,039 $49,843+$11,039=~$60k saved which is 20% of what they were short, if they had started with April 1st's pricing.

If you can reduce the AWS $400k by 40% due to the reduced pricing for EC2, you'd still fall short.

Why wouldn't they put all of that in a Backblaze Storage Pod? 2 months of AWS costs could've built more than 2 pods and stored everything with plenty of space to spare.

Price out the backup solution for that back blaze. Then make the backblaze geograohiclly redundant. Then write the system to keep them in sync. Then add versioning. Then pay for someone to manage it. Now you are getting towards the real cost of that backblaze to replace S3.

>Then write the system to keep them in sync. Then add versioning.

Are there any ready-to-deploy solutions for this?

That's no different than startups of the past having to pay thousands to millions of dollars to setup servers. These days those same servers can be setup in minutes for a fraction of the cost.

Timing is everything. Sometimes the current market pricing for a commodity is too expensive to make your business viable today but in the future that will not be the case. Just depends how long that will take.

Fully agree. Most of what makes any startup work in the first place is that time + money + technology have finally aligned for them to start their idea.

If S3 storage pricing was the issue for this startup they could always have gone with their own dedicated servers for storage for a fraction of the cost. For that reason, I doubt a lower S3 cost would have prevented them from shutting down.

Fraction of the cost long-term. Not short term, where they found themselves.

Renting dedicated servers with lots of disk has typically been vastly cheaper than S3 "forever" on a monthly contract basis, so there are plenty of options before going the route of bare metal in your own rack.

But even renting space by the 1U and leasing servers directly adds surprisingly little overhead.

http://blog.backblaze.com/2014/03/19/backblaze-storage-pod-4...

I'm not sure where you get that data, but it doesn't match anything I've found in the 27 dates centers we do business in. Storage, versioning, redundancy, durability and geographic availability. That is the core of S3, not just storage.

The data centres makes little difference in this case. You can fit far more storage than what Everpix used comfortably in a half rack (e.g for density I can buy 5U servers storage servers with up to 48 x 4TB drives). In London, that runs me at most about $1k/month per site. Three sites for $3K/month. 400TB worth of off the shelf storage servers would run us ~$1600/month without even shopping around, in leasing cost per location, for ~$8000/month for triple redundancy on top of raid. Then there's bandwidth.

400TB even at the Reduced Redundancy Storage pricing is ~$20k/month up until the price drop. With the price drop it looks like it'll be more like $9300 with RRS, which is actually getting cheap enough that it's worth it in some cases, to avoid having anyone spend their time on managing anything. The gap will increase again once you factor in request pricing and bandwidth, though, as even my standard price list prices for bandwidth would add up to less than the cheapest published bandwidth tier at Amazon with just a tiny little bit of hassle (BGP setup to multihome each sites with 2-3 different transit providers with different coverage - lowers cost by letting you pick a transit deal that does not give full access to the internet at very low cost coupled with a full price full coverage one)

Or I could spend about $12k/month at Hetzner for the same storage + an aggregate stated "guaranteed" bandwidth of about 2Gbps per location at no extra cost. Again, without shopping around. The moment you do more than about 20-30TB transfer/month, Hetzner becomes cheaper than S3 with that setup, and probably at about 50TB+/month it'd take effort to beat them when buying my own transit too.

You're leaving out the part about the software and operational support to handle failover, data integrity checks and correction, etc. You're not just tossing disks in a rack but managing a highly-available, highly-durable storage system on top of them.

You can definitely beat those rates at large enough scale but for a startup you're talking about dedicating a non-trivial chunk of your skilled technical staff's time in the hope of breaking even. That's an obvious move for Backblaze or Crashplan but for other companies it might be a dubious investment versus nailing the core product.

Not sure how you can compare your own servers to s3 - s3 has redundancy built in that can survive the loss of 2 data centers. I'm pretty sure it would be more expensive to roll your own version of that...

See my comment elsewhere with some numbers. You can beat S3 even with the new pricing in many cases. With the old pricing its trivial.

With replication to 3 data centres.

There is no way to replicate S3s full service level at a fraction of the cost. You can store the data pretty cheaply, but you can't replicate the full service level anywhere near as inexpensively. Especially the redundancy, vesioning and durability.

Its one of the factor, but it's not the only thing which can put them in the black.

A miss is as good as a mile.

As an AWS consumer, I'd like to thank google's cloud team for making this announcement in early 2014 possible.

This must be an incredible benefit to companies like Imgur or Dropbox where a significant portion of their costs must be S3 storage.

I wonder if any company using that much S3 space wouldn't have already negotiated a better deal, possibly beating these prices.

It must, but I assume once you get to a certain level of usage then an account manager / AWS gets you a slightly better than the publicly published deal. Is that the case?

Also, I wonder if EC2 is up for a price tweak soon too?

EDIT: Ah - https://news.ycombinator.com/item?id=7475284

Yes, the big users have negotiated lower prices that aren't listed on the site (confirmed to me by a very large S3 user).

If you have a direct-billed account, and are spending more than 1,000/month, you'll get an automatic price break. At my current $DAYJOB, we're billing around $15-20K/month (depending on traffic) in Amazon Web Services charges, and I believe we're getting somewhere between a 10 and 25 percent price break.

I remember reading a while ago that Imgur uses Edgecast as their CDN and not Cloudfront. See: http://readystate4.com/2012/08/14/great-post-on-imgurs-stack...

My guess is that they are only using AWS for serving the site and handling the uploads, resizing etc. and they are using another provider for the storage of the images (or they are just directly hosting everything with Edgecast and not using them in Pull mode).

Like people have mentioned above, you can get a bunch of dedicated servers with heaps of storage for a fraction of the price of S3. You loose the flexibility and instant scaling, but the fact is these servers would be used more of a backup and would only be hit by Edgecast to upload them to the CDN.

Benefit to them, doubt we'll ever see the savings in the case of Dropbox

Dropbox doesn't pay the rates you see on the S3 pricing page. They currently pay lower than $0.03/GB.

I hope we see a price reduction for the 100gb @ $10/month account, for something closer to $5/month

Is Dropbox not having a $5/month plan a decision based on storage pricing?

A reduction in the cost of goods sold does incentivise businesses to apply price discrimination without a negative impact on margins - a $5/month service would target a big, untouched, monetisable market.

I would be surprised if they didn't reduce their prices, there have been too many competitive storage pricing reductions recently for them not to.

Nothing should ever be announced or "effective" April 1.

I'm set to move into a new apartment building that is opening April 1st. I'm still not entirely convinced that I'm actually going to get my keys...

I launched a company once on April 1. Realized the date after we'd announced to everyone. We were worried about it impacting launch (is this a joke?), but all was good.

Apple was founded on April 1st, 1976

April 1 is not celebrated everywhere in the world.

I learned that April 1 jokes on the internet are often taken the wrong way even when there are ample reasons to realize it is a joke.

See http://osdir.com/ml/finance.ledger.smb.devel/2008-04/msg0000...

There is precedent around making big announcements on April 1st, with full understanding of the impact of announcing on that day:

https://www.google.com/search?q=founding+date+apple&oq=found...

http://en.wikipedia.org/wiki/Gmail

Storage is cheap, but data transfer out is still $0.12/GB, over 10x more expensive than other options.

Not to disagree, but what other options? Google and Rackspace charge the same. DreamObjects charges $0.05/GB in transfer, but still charges $0.05 for storage. Azure is still the same $0.12/GB.

I don't disagree that $0.12/GB could be considered expensive, but S3's competitors seem to be charging the same.

I actually use DigitalOcean as a cheap download server for my product. $5 gets you 1TB, then its $0.02/GB

How much bandwidth are you using? I'd wonder if they throttle you at some point. I'm sure they're not expecting people to use that whole TB.

I use multiple instances (20-40) of 1 TB bandwidth pretty regularly with no throttling as of yet.

Yeah I think they realize, like most vendors, that most people don't even come remotely close to using that much bandwidth. Even using them as a proxy for nextflix will probably just end up being 300G a month.

On the other hand, S3 <-> EC2 transfer is free of bandwidth costs (there are still per-request costs though).

Also, inbound traffic is free.

Within the same region, yes. Not in the same region, then whatever the inter-region bandwidth costs are.

This was my take-away from the Google Cloud pricing changes yesterday. By reducing CPU costs, my bandwidth just becomes even more the dominant cost, so further reductions help me less and less.

AWS users will end up with the same problem in time. I wonder why the bandwidth costs are so subborn, does the cost price per traffic unit really never come down, or is someone (some people) in the infrastructure layer(s) screwing us?

At the scale large providers operate at, they don't pay per GB, they pay at the 95th percentile for Gb/10Gb links.

It's the same way you pay a fixed hourly rate (in most cases) for your retail electricity, and your provider is paying wholesale spot market rates that can fluctuate in ~5 minute windows.

TL;DR You're paying for consistency and abstraction away from the underlying IP transport costs.

Transit costs less than $1/Mbps/month, which works out to only a few cents/GB. It still looks like pretty high markup.

You're assuming its only the transit cost (as does my original cost); it could also be to discourage use of S3 for mass public serving due to architectural constraints.

My understanding is that S3 is near indestructible from a serving perspective, but it may be computationally expensive for AWS to scale up/scale down for bursty/peaking outbound traffic serving needs.

Another possibility is that Amazon doesn't charge for inbound, but charges a premium for outbound to balance their traffic ratios so they can peer with providers vs having to buy their transit. The free inbound traffic to S3 offsets their outbound traffic not only from AWS, but their consumer-facing web properties.

As always, just my assumptions/observations.

In my opinion they're doing that on purpose to discourage people from using S3 as both a storage and a transfer method.

Storage and Delivery should be considered separate, as there are benefits to it being a separate service. If you throw a CDN between your customers and S3 you'll not only get a significantly lower cost, you'll also give your customers much better performance.

In that case, where's the price drop on Cloudfront?

Personally I would not use the Amazon CDN. It's not as cost effective as others, and CDN services are basically commodified at this point.

Would you happen to have recommendations for other CDN services?

Akamai on the high end, Fastly for a startup or mid-sized business.

looks like that fastly and cloudfront pricing is not that much different, they apply pretty much the same prices for traffic and requests in each of their zones.

http://aws.amazon.com/cloudfront/pricing/ https://www.fastly.com/pricing

We use cdn77 and are happy with the service. They're cheap and they properly route Google DNS users (lots of them), since they support edns-client-subnet.

I just did some price comparison. It looks like cloudfront is actually the same price as data transfer out of S3, but in addition you have to tack on a $.02/GB fee to get the data to your cloudfront edge nodes.

If I am wrong about this (and I hope I am), where can I find more information about that?

Thanks!

No, you're right. I use cloud front and you have to pay for the day at to go from S3 to cloud front servers where it gets cached. Then the costs are $0.14/gb in the cheapest region.

If your cache gets deleted or expired, you'll have to pay to move the data (at s3 rates) to the CDN again when someone requests it.

Cloudfront actually becomes cheaper than S3 once you start scaling up to large amounts of data, plus you get the added speed benefit.

The bandwidth cost from AWS to the internet is the same for EC2 and S3. They aren't discriminating as far as I can see.

Which is why you should be relying heavily on a caching layer outside of AWS. You're not serving objects directly from S3, are you? A big no-no if you care about latency at all (S3 from time to time will take upwards of ~500ms to serve HTTP GET requests).

EDIT: Example: We use Akamai as our caching layer, as their bandwidth costs are much less than Cloudfront (their analytics are better as well, and they don't charge for invalidations like Cloudfront does). We use S3 as the origin as its a lower cost per GB than Akamai's NetStorage, and the S3 API is superior to work with compared to the interfaces exposed by NetStorage (scp/ftp/rsync).

What "other options"? You mean sketchy "free 9000 exabytes/month of bandwidth" low end hosting places like hetzner? Those are not alternatives to AWS or azure or google cloud. And those bandwidth costs do not reflect reality, they assume most people use almost none of that bandwidth. The bandwidth still costs the same, just the big users are getting the small users to pay for their bandwidth costs.

That doesn't change the fact that you pay much less for much more.

On a completely different service. Which is the point. I get more food for $10 at burger king than I do at a hotel, but that doesn't make burger king a replacement for hotels.

Is Amazon's bandwidth more delicious than Hetzner's bandwidth for the same price? Are the servers more nutritious?

It isn't the bandwidth that is the difference, that's the point. How are the beds in Burger King?

There are lots of other options. While they may not all be "cloud" providers, there is a wealth of VPS and dedicated server providers in almost all countries you can name.

I have no idea why you are picking on Hetzner in particular. They offer clearly defined bandwidth packages with their servers (20-50 TB) and charge for overages (0.2 cents per GB).

Hetzner is perhaps not what you would call a premium provider, but their prices are about in line with industry wholesale rates. For example a very high quality provider that competes with Hetzner in their home market charge 0.4 cents per GB and they are happy to deliver any traffic you can serve at that price point.

>While they may not all be "cloud" providers

Then they are not "other options", that's the point.

There most certainly are other cloud providers out there, but it would help if you defined what you are looking for instead of conflating issues.

Webhostingtalk is a good resource for the curious.

>There most certainly are other cloud providers out there

With similar pricing to amazon. Are you not reading any of the replies you've gotten?

Yes, I've read them and responded accordingly. Perhaps you could do the same.

There are cloud providers with cheaper prices than Amazon. Please feel free to read up on the alternatives.

The whole point of this exchange has been to point out that wholesale prices are far less than what Amazon charges. Some cloud providers are more prone to follow the wholesale cost level and others are not.

If you want to discuss alternatives to Amazon then first you have to define what cloud features you want/need.

There's a fair number of hosts where you simply get a 10, 100 or 1000mbps port and then use it for unlimited traffic. As long as the provider doesn't pay per MB, there's no issue in filling it up. The main difference with places like S3 is that S3 has great peering arrangements all over the world, whereas such a host probably has worse peering to some places.

But I did ~50 TB of traffic per month at various hosts for <$20 without any issue. Just choose them well, and know exactly what kind of connection they offer.

I hesitate to even post, as offsite backup is very different than the use case for S3, but we're the opposite...

That is, our storage is expensive (10c per GB, per month, with the HN-new-customer discount) but our usage/transfwer costs are zero.

  ssh user@rsync.net s3cmd get s3://rsynctest/mscdex.exe

interoperability!

Lets put some math on the network costs. An oft quoted cost is $1/mb at p95 for decent transit. 1 mbs * 2,592,000 seconds = 2,592,000 mb per month, / (8 * 1024 gigabytes) = 316GB transferred per month. Dividing our $1 / 316GB = $0.0031 per GB transferred _at a constant 1mbs_.

However most networks have sine wave pattern diurnal utilization. A common peak to average ration would be 3:1. Our $0.0031 is effectively the rate for an average of 1mbs. To get that average rate we'll be paying about 3x, or $0.0093 per GB transferred. So we're paying $0.01 per GB for the transit alone. This is also in a major metro area in North America. Transit costs can easily be 3-10x in EU or 10-20x in AP.

On top of that there's capex and opex for that 10gb port. Assuming you've got some decent scale you're going to get 96 10gb ports in a $300,000 to $500,000 router, or $3-5,000 per 10gb port. You need ports both north and south, so you get 480gbs throughput per chassis. But you can't run above 80% util, so call it 384gbs of peak throughput. Opex for rack position, data techs, power, etc will cost $2,000 to $5,000 per month. Amortized over 36 months let's call it $10,000 to $17,000 per month all up.

That 384gbs peak throughput is actually more like 128gbs averaged over the month. At 128gb/s average we transfer about 41,472,000 GB per month, or $0.00024 to $0.00041 per GB. Now double that cost 'cause routers actually come in pairs for fault tolerance. So we're paying $0.0005 to $0.0008 per GB of transit connectivity.

And how many hops are there back to your application? I'd hazard around 6 by the time you go transit -> edge -> border -> core -> dc -> agg -> tor. Paying for that internal network infrastructure is going to add $0.0030 to $0.0050 to our bill. All up that gets us to $0.0128 to $0.0151 per GB transferred from our host to a third party network.

Expanding the error bars lets round to $0.01 to $0.02 of network costs per GB that we deliver. To get these prices you're going to have a capital budget of tens of millions per year. OpEx would be at least $500K, or more likely a million plus, per month. If you can actually beat my pricing I'm hiring and or would like to subscribe to your news letter.

Apologies if I missed something in the math. I had to do some mental acrobatics to go from dollars per gb/s to dollars per GB/month.

Given costs of storage and data center operations, what's the lowest, commoditization-bound on standard Amazon S3 prices? Now they're reducing to a narrow $27.50-30.00/TB-month, from the current $43-85/TB-month.

I've seen Internet Archive and Backblaze estimates that indicate $60-100/TB-storage installation and $7/TB-year in power. If drives last 5 years, we could expect a commodity cloud-storage price around $20/TB-year? One order of magnitude to go.

a while back i wondered about the same thing ( https://news.ycombinator.com/item?id=2787770 ). It did occur later to me that S3/Google Cloud Storage and Archive/Backblaze are really not the same thing. S3/GCS is FAST.

Backblaze, for example, has two gigabit ethernet ports for their new 180TB pod. That means, even assuming no overhead, copying 1 POD to another will take ~9 days! Even were they to add additional/faster ports/interconnects, the 45 drives in a POD are still sitting on a PCIe 2.0 8x, thus with a theoretical throughput of 4GB/s, thats still half a day.

Now, I do not know how many drives Amazon/Google load into their S3/GCS servers, but my (dayjob) live storage servers rarely have more than 4 - 8 drives each. Even if we assume 8, thats a 5-6 fold increase in theoretical throughput - and THAT is why S3/GCS will never be in the same pricing zone as Backblaze.

S3/GCS, even Backupsy, DO, etc are all "live" storage solutions, and seems most are converging to ~3 cents/GB-month. Backblaze is "archival" storage, and will (probably) continue to be cheaper by a factor of 5-10 (or its won't be worth the hassle).

Amazon is well known for losing money to capture market anyway, so I wouldn't put break even as their limit.

Now for those without context, that was to match Google Cloud Platform's aggressive offering they announced just yesterday. Now it's Windows Azure's turn I guess.

Much expected after the GDrive prices got slashed.

Currently I am only using S3 for hosting PDF or other big size files which are downloadable by the public, since the bandwidth is pay-as-go.

However for regular storage, Dropbox/GDrive are still the primary choice, due to the fact, the data sync across devices, speaking of sync, what is holding Amazon back from a similar app for S3?

Gdrive is not a competitor to S3. Google Cloud Storage is.

I should have made it clear, thanks for pointing it out. I guess this was in the wake of Google Cloud Platform pricing going down yesterday?

Also Amazon's Gdrive competing equivalent is cloud drive.

I think they were planning to announce a price drop anyway as they are doing their annual cloud shindig in SF today.

There is Amazon Cloud Drive which is the consumer focused storage which is similar to Dropbox and Google Drive. It has a sync app. The prices for Cloud Drive haven't been cut; they are still $0.04/GB/mo.

thanks.

This almost brings it down to where I can start backing things up onto S3. Currently my home NAS houses just under 500 GB's of stuff. My requirements to back this up offsite would be (1) full encryption of each file and filename using a key only I hold and (2) reasonable per-month price to where I cannot justify just buying another box with a bunch of drives and sticking them at a friend's house.

I believe I can solve the former with S3, though I still don't know of a turnkey solution. The latter is not quite there but this update brings it a lot closer. At this point it would cost me $15/month or $180/year. That's not terrible, but for that price I can easily have two 2TB WD Red drives. The box to house them would cost just a bit more since I need no horsepower, but just enough to run ZFS.

Glacier is a more attractive option, but the fact that the price is so complex when transferring data out of it, I'd be looking at taking months to restore everything just to not pay to dollar for it.

Check out git-annex (and the corresponding assistant). I haven't used it yet, but it supposedly supports S3 and glacier, and hard drives and will tell you where each file is backed up.

Interesting option. I might use this inside the LAN to manipulate the files on the NAS. The current setup is much simpler though: files that live in ZFS and incremental backups every day and every week.

Check out CamliStore at http://camlistore.org/. It's far more than backup, but it accomplishes that as well and supports S3 as well as Google Cloud Storage.

Very interesting. I will check it out further, thanks.

Google Drive would be $120 / year for your 500 GB.

Interesting. I will have to see how this will handle encryption and large files.

In my experience - and from what I've seen in my subsequent frustrated research - Google Drive struggles with large files. I had a 9Gb VM that I split into 2GB pieces, and I still couldn't get one piece uploaded in a 24-hour period. Also, there is no progress display and it seems that a simple 'pause' of the sync restarts the process.

The GD client / back-end and large-file handling seems to be a long way behind the alternatives, sadly.

BackBlaze is pretty popular with the HN crowd and their pricing is still $5 a month for unlimited backup space.

Back blaze is out for two reasons: no Linux support and unlimited backups. I want to know exactly how much storage I can use up before hitting some abuse policy.

Isnt backblaze unlimited storage? I thought that was their main feature.

IgorPartola means that it is exactly this "unlimited" policy which chills her. She speculates (or knows) that there is some unspokenen abuse limit and she fears that she might hit it at some point.

I don't think this is relevant for you if you have less than a few Terabytes of data.

I'd recommend Arq (a great backup product for OS X).

It requires some technical chops to setup, but I doubt that would be a problem for you (or other people on HN).

http://www.haystacksoftware.com/arq/

This seems to be for backing up workstations. I have that covered already with TimeMachine and Ubuntu's backup services to the NAS box. Now I just need to back up the NAS itself. Thanks for the suggestion though.

You can back up your NAS to rsync.net.

I could but for $1800/year I could also build at least 8 redundant NAS boxes with 2TB storage and stick them at my friend's houses. I could also just pay $180/year for S3. Your offering is VERY expensive though I am sure quality is top notch.

FWIW, the HN new-customer discount brings that down to $1200/year.

Probably ... 2x the price of S3, given the cost of data transfer which, for us, is zero.

It may not be for you, but some folks find phone support, 7d+4w snapshots, straight-up, native unix interoperability and 2 free physical delivery events per year to be compelling.

Cheers!

Waiting for Azure response, should be any minute now i suppose.

Same date as Google's cloud storage price changes.

Is it reasonable to expect Paas service providers like Heroku and Engineyard also to cut their prices, or is it asking for too much? On one end, it seems logical to me because their services are built on top of AWS for the most part, on the other, I am not sure if this might happen since their selling point is not the Infrastructure underneath, but rather the convenience and ease of getting something up and running. I guess we'll just have to wait and watch.

The linked page is only describing a price change in S3, not all AWS services across the board (most of what you're doing with Heroku would be incurring EC2 costs).

Thank you for the clarification, looks like I commented on the wrong thread xD

Am I reading this right that the new pricing is going to be 3 cents per GB and the old pricing was 8.5 cents per GB? 2.8x drop in price! Wow!

I hope this causes services such as Tarsnap to reduce their prices. Doing a full off-site backup, including pictures, is still pricey.

Glacier storage cost isn't changing, just adding one decimal (extra zero).

nice to see that "GovCloud" is 4x more expensive

http://aws.amazon.com/govcloud-us/pricing/s3/

the $12,000 hammer reinvented!!

While I don't know it's x4 worth it, I can personally testify to how expensive anything FIPS 140 is to get done..

"Because the AWS GovCloud (US) Region is physically and logically accessible by US persons only and also supports FIPS 140-2 compliant end points, customers can manage more heavily regulated data in AWS while remaining compliant with federal requirements."

http://aws.amazon.com/govcloud-us/

Keep in mind that this means they're operating one or more facilities exclusively for GovCloud. Since these facilities are almost certainly smaller than the rest (and, as you mentioned, they're more work than normal), it only stands to reason that it'd be more expensive to use.

I'd like to propose a new rule for Hacker News: only if you have built the thing you're saying someone should save money by building themselves, may you say the person should build that thing.

Thanks Google!

This is amazing news. I have a image gallery site with ~500,000 images. It currently costs me $39 a month and now I'll get to spend even less, no idea how they do it.

I hope these AWS pricing changes are not an early April Fool's joke. The S3 pricing in particular will be cheaper than the current pricing for the > 5000 TB tier.

It's pretty interesting that there have been no update to the Glacier prices as of yet. I wonder what could a fair point price be ~ 1 cent per TB ?

I wish the cost for PUT requests would drop.

Unfortunately, PUTs are just more expensive computationally than reads. You could always use a caching layer and only put for persistance.

Is it just the cost of storage that dropped? Bandwidth is still a major burden.

We picked a good time to start using AWS! Nothing but positive things so far.

April fools!

(mindblown)

Man, I guess nobody likes even a little humor around here.

I too, took some punishment for making a joke. HN is a dead serious place, I guess.

Jokes are welcome, but they have to actually be funny.

But jokes do have their place right? They brighten someones day, for example.

I propose someone makes a subsection of Hacker News dedicated solely to jokes or funny stuff.

Or someone create a new site... named.... Hacker-News-Chan?

Just be happy that you didn't get hell banned because someone didn't like your joke. HN is worse than reddit sometimes.

Now I know why the soap in the toilets is not getting refiled.

Reply on news.ycombinator.com