AWS drops bandwidth pricing | Hacker News Reader

AWS drops bandwidth pricing

aws.amazon.com

271 pointswerner93 comments

Making all in-bound traffic free is a super-aggressive (and much appreciated) move.

As blhack pointed out Voxel's per-GB rate[1] before AWS dropped was extremely competitive, but they charge for in and out-bound data. AWS, after the 1st of July will only charge $0.12 for out-bound data and $0.00 for inbound data, effectively making it something like $0.06/GB compared to Voxel (I'm hand-waving this a bit to make a point).

Also as wiradikusuma pointed out, this comes right on the heals of Google's App Engine pricing structure change[2] to be more business-friendly (read: more expensive/more predictable billing) that upset smaller shops and individuals.

As someone who reads most of the AWS forums every night, I would say overall that Amazon seems to be responding more quickly to low level failures that used to run rampant on the system (although US-EAST still has more failures than any other region. I guess due to overload). They seem like they are hitting faster/smoother, sounds like a good time to push forward and grow which I imagine this move will help do.

Getting a little excited to see what the price decrease for per-GB billing on S3 will be in the coming months following this up (my assumption).

[1] http://www.voxel.net/pricing

[2] http://www.korokithakis.net/posts/app-engine-pricing-changes...

It also makes it really appealing for a lot of data-analysis use cases, which are increasingly being done on EC2, because it's a nice way to burst up to a temporary cluster. That use case often has large inbound traffic, as you pull down datasets (say, some genomics data), but then comparatively little outbound traffic.

Indeed, this is exactly my case: 10-100's GB of data distilled down to 1GB output at most, including extra diagnostics which are not always needed.

One possible advantage for Amazon could be, at least in my use case, that I may be inclined to spin up instances for less-CPU-intensive tasks. Also, being able to rely on the widely available remote access from EC2 is convenient.

I suppose if it gets me spending more time on their CPU cycles, they've picked up business. Interested to see where this policy leads.

for 99% of websites...$0.12/gb OUT and 0.0/gb IN will not effectively make this $0.06/gb.

Can you elaborate on this? Not disputing your point, but it would be nice if someone could jump in with some real world numbers or a rule of thumb.

It's because of the ratio of reads to writes. For many applications, especially in social/content, the creation traffic is an order of magnitude or two less than the consumption traffic. So free inbound bandwidth is a bit of a rounding error.

Here's a somewhat old article that captures the rule.

http://www.guardian.co.uk/technology/2006/jul/20/guardianwee...

Goto facebook, or SO, or HN or youtube or almost any site...open firebug or whatever, and calculate the amount of inbound traffic to the outbound traffic (keep in mind that this is reverse for you and the host). You should notice that your outbound traffic (their inbound), which is mostly short GET headers, is like 1% of your inbound (their outbound).

Sites where you upload a lot of content, like YouTube, still have much, much more reads (outbound) than writes (inbound). Probably by an order of magnitude.

Systems like Dropbox would benefit - though they were already getting different pricing I'm sure. But this is really a tiny fraction of how most online systems work.

Latch, very fair point. I did a bit too much hand waving there, but maybe more like $0.10/GB would be more accurate.

The pricing change is better understood with the tables here which include a "previous" column:

http://aws.typepad.com/aws/2011/06/aws-lowers-its-pricing-ag...

This is probably a move in response to Windows Azure dropping its inbound data transfer rates to zero last week. When I was back in Windows Azure, we would often see AWS try to do a price-match whenever we changed prices drastically.

I know Amazon pays very little attention to competitors. However, their customers might have asked them to lower prices in response to Azure.

I doubt they process customer feedback in the matter of a few days :). Besides, there's a history of AMZN doing things just before or just after a Windows Azure announcement. See their announcements always timed the week before MIX or PDC or the response to various other pricing changes, etc.

Trust me, Amazon keeps a very close watch on the competition. :)

Can't comment as to what AWS does, but Amazon itself pays a ton of attention to competitors, employing things like screen-scraping bots to keep track of competitors' pricing.

I'm blown away. This radically changes the cost for one of my core products (automated browsing from EC2 machines). I've thought for a long time that EC2 was getting comparatively very expensive for bandwidth (simply not decreasing their prices). I thought they would have to change it, but I didn't expect free!

I wouldn't be surprised if they put a fair use clause somewhere. Otherwise you could hypothetically run a fairly resource intensive bot that scrapes data 24x7 using up aws download bw for no cost.

Instances are already IO bound/capped.

For example, if you start up a micro instance, you'll only be able to download at like 200kbps or something.

So I'd assume the connection speed is linked to the size of the instance, and the bandwidth cost is integrated into that cost somewhat.

any more info on this? I had no idea this was the case

I don't think this is accurate. I have seen download speeds of around 6/8MBps on windows micro instances. The lowest I got was around 3MBps. But I only tried with downloads of around 30MB files at the maximum.

The io is burstable. Try doing a sustained large download and you will see the speed throttle down

No, I'm seeing a sustained 6MB/s on this US-EAST micro.

This

  wget -O /dev/null http://cachefly.cachefly.net/100mb.test

gives me about 30 MBytes/s from a micro instance in EU-West

Sorry yeah it looks like my test was disk IO bound rather than network bound.

I don't think AWS needs a fair use clause, because their cost for incoming bandwidth is virtually zero (since they have to buy symmetric connections) and if you're sucking in a lot of data you're probably paying them a reasonable amount to store it, process it, or both.

My blogpost w background information: http://wv.ly/iLDaqu

Direct link: http://www.allthingsdistributed.com/2011/06/aws_bandwith_pri...

I wonder if this was in reaction to the same pricing change from MS Azure http://blogs.msdn.com/b/windowsazure/archive/2011/06/22/anno...

Amazon is now the #1 choice for web scrapers everywhere :)

I thought most AWS IP addresses were blocked by most admins. Does any one have experience scraping using Amazon.

We have a pretty big e-commerce scraping project, and we don't really run into many problems regarding blacklisted ip's. There are a few sites who we get consistent bans from, but with elastic ip's it's pretty much a non-issue. I have yet to see a site that ban's all amazon ip's.

It is annoying when it happens and you're on something like Heroku.

I run http://whirlpool.net.au and I religiously check the Amazon EC2 forum announcements[1] for new IP ranges to ban.

[1] https://forums.aws.amazon.com/ann.jspa?annID=1030

> I run http://whirlpool.net.au

An excellent and very useful forum! It seems like whatever topic I'm searching for, google(.com.au) returns a useful result on your site.

Would you mind telling why?

Why?

Name me one good reason. Name me one.

Shitloads of rogue bots doing "social media monitoring".

Shitloads of rogue bots stealing content for black-hat SEO.

Shitloads of rogue bots harvesting email addresses.

Shitloads of rogue bots submitting spammy replies.

That's unfortunate. I don't know what the answer is, but real people do route their traffic out of AWS endpoints.

Far fewer than you might think.

So maintain a blacklist of elastic IPs. If it's too big for you, make it a community effort.

Those are bad reasons to close your site to all of AWS.

As nupark2 mentioned, there are legitimate users routing traffic through EC2, even some bots that you'd want to visit your site. Archive.org comes to mind (many of there scrapers are or were behind AWS). Closing your site or app to a large swath of the web is the wrong solution. It's like killing a spider with a bazooka.

Unlike the assumptions you're limited to making, I know how much of my AWS traffic is human, and it's really very very very small. The sad reality is I'm sick and tired of rogue bots, and the tiny sliver of collateral damage can fill out the CAPTCHA validation every so often.

(I also blacklist GWS, rackspace, linode, softlayer, reliablehosting, ovh.net, node4, netdirect, layer42, all TOR exits... it's actually a pretty huge list.)

I whitelist archive.org, and they've never hit through AWS.

I don't scrape, but I use an EC2 micro instance as a VPN endpoint when I am on an unsecured wifi network.

It's a little annoying that some sites block me solely because I'm coming in from AWS (StackOverflow, I'm looking at you).

I suggest using prgmr.com instead. $6/mo is worth a Xen instance with IPv6 and everything.

Most admins? Most admins probably don't even know what AWS is. Most admins probably don't even block a single IP. I would be surprised if as many as 0.1% of admins block AWS.

In my experience no UK newspapers block AWS IP addresses.

I usually use Tor when I scrape anyway :)

Free (or cheap) inbound bandwidth is pretty common from colo providers.

For another comparison, voxel.net (which serves imgur):

http://www.voxel.net/pricing

$0.10/GB up to 40TB

$0.07/GB up to 500TB

$0.05/GB >500TB

This looks like the cheapest "real" CDN I've seen. Awesome :) Not that I need it [yet], but here's to hoping :)

These prices apply to inbound and outbound traffic, Amazon made their inbound traffic free.

Isn't the vast majority of traffic outbound?

A lot of traffic is fairly symmetric: chat, email, dropbox with two clients.

EMR jobs can also have much more inbound traffic than outbound.

Not for apps that back up to S3 :)

I use CDNetworks, and at a 50TB commit I'm getting /much/ better pricing than that... I've even been quoted better pricing than that by Akamai (which people in this industry view as notoriously expensive). I might encourage you to drop the quotes around "real" and look at actually real CDNs (the kind you have to call and negotiate pricing with): they aren't as expensive as you apparently think they are.

Are you seriously suggesting that imgur, which is currently doing almost 1PB/month, isn't using a "real" CDN?

Quotes are there because I've heard of people buying up "unlimited bandwidth" shared hosting packages, and using them as a cheap "CDN".

No, just that MrGrim is getting ripped off because he doesn't know the industry.

grim isn't paying that much for voxel. Voxel cut him a ridiculous deal in exchange for the advertising they get out of it.

We're talking about them right now. The only reason I mentioned them is because I heard about them in relation to imgur.

Dropbox must be super happy that half their bandwidth costs have disappeared!

This might be very related, actually - Dropbox just sent out an email announcing downtime for a move to a different data center.

Also, backup solutions based on S3. Most of their traffic is inbound.

About 90% of Tarsnap's bandwidth is inbound... but most of Tarsnap's AWS costs are for storage, not bandwidth. Backup services tend to store a lot of cold data.

As a Tarsnap customer, would this allow you to stop charging on sending data to Tarsnap, or does that still cost you somewhere in the lineup?

Tarsnap's upload costs cover the actual bandwith costs, the per-operation cost of S3 PUTs, the SimpleDB operations for checking account balances, some data storage (when you delete data, you stop paying for it; but I pay until the garbage collection job reaps those blocks)... there are lots of costs. The bandwidth was a significant part of it, though.

I need to crunch some numbers and see how the economics work out before I can commit to anything, but I'm certainly looking at adjusting the pricing.

I assumed there was more to it than just bandwidth costs, just wanted to get an official answer. Thanks.

Upload is probably much less than half of dropbox's bandwidth. If multiple users have the same file, it only needs to be uploaded once; but it needs to be downloaded every time someone syncs a directory containing it to a new machine.

Now if only Google App Engine would follow suit...oh wait they raised prices =/

This is big. It makes it a lot cheaper for a busy site to keep an up to date mirror at Amazon on standby for use in emergencies.

this should put some pressure on recent Google App Engine price increase (fingers crossed)

Everytime I see a price drop my first though is always, "I wonder how much this increases Dropbox's revenue?"

I don't even remember what the old pricing was. For the first (non-free tier), was it at 0.18? 0.12/gb is getting pretty cheap for non-bulk bandwidth of this quality.

edit

above poster has link showing it was 0.15

For someone inexperienced in this market, why wouldn't they just say free? I've never heard someone sell something for "$0.00"

Free is better for singular marketing messages. But for tabular data, when you're comparing different points, keeping the unit consistent is better.

The timing for this Django deployment script couldn't have been better: http://news.ycombinator.com/item?id=2700120.

I was going to look into deployment scripts for App Engine, but Amazon makes it more compelling to use AWS.

Good lord, about time. Why was incoming bandwidth ever on the list?

Because incoming still takes up bandwidth so it makes sense to count it is such. I however do remember it being free at some point in time.

It's in their best interest to get as much in as possible considering they charge for storage and deletion.

We're looking for even cheaper bandwidth for streaming audio. Can anyone recommend a vendor to provide 150mbit+ bandwidth on a vps platform or rented server platform?

Why is bandwidth pricing higher in Asia despite bandwidth rates there being among the highest in the world?

Your question answers itself.

I really don't know how much more to add, but I'll try.

Transit (bandwidth) costs more there because:

  1) Networks peer much less openly.
  2) This is mostly due to the presence of state-run monopoly telcos.  
  3) That's why BW costs more.
  4) Therefore, it costs more for AMZN to get transit in Asia.
  5) This is why it costs more for AWS customers to use bandwidth.
  6) This is why it costs more for you.

It's more expensive per mbps in USA compared to Asia [0].

[0] http://www.billshrink.com/blog/5787/internet-penetration-cos...

Edit: I'm assuming the data transfer rates apply to data leaving the data center in Japan -> Any computer, not USA region -> Japanese computer.

That's not talking about transit -- that's talking about consumer Internet connections, which are surprisingly cheap despite the same monopoly infrastructure but largely due to government price controls.

That monopoly infrastructure, however, is what keeps outside providers from making inroads -- you don't see anyone but SingTel in Singapore, for instance. Or anyone besides Malaysia Telecom in Malaysia, or NTT / KDDI / etc. in Japan.

Ah, makes sense, sort of. If there's government price controls, why was the internet monopoly company still increasing bandwidth when they are already leaders in worldwide bandwidth?

Great engineers? I have no idea. They make us look bad in the states though, so I say keep it up -- hopefully it'll motivate our guys to speed up my last mile.

In HK, I pay around $22/m (USD) for 100mbps. I can download from steam or apple at 5-6+ MB/s. I can download torrents at 10MB/sec. However, when I download overseas (which is most of the time), I might get 200KB/sec (unless it's from a serious player in CA).

The point? The high speed you hear about in Asia is a half truth. When you consider that some of the Internet doesn't exist here (Netflix, ...), high latency (for gaming) and geo-locked content (Kindle, ...), it's even less than a half truth.

Anyways, where NA has poor last-mile infrastructure, Asia has poor intercontinental infrastructure. It's currently a scarce resource, thus prices are high.

Asia has the intercontinental infrastructure, but the costs to connect are too high. SingTel, etc. all run monopolies that keep foreign networks out. Peering is a joke.

Have you tried something that doesn't have the overhead of torrents and can make many concurrent connections like a local HK usenet server?

I've gone up to 10MB/sec...which is close enough to the theoretical limit of 100Mb/sec. Locally the product works as advertised. But only locally.

> But only locally.

Fair enough. I use Usenet almost exclusively at this point so I would love that.

It's true though, I had this illusion of people with that kind of connection being able to connect to anyone on a quality server instantly.

ooh "AWS drops bandwidth pricing" come on please be true! darn it just inbound.

It's not just inbound, see the "previous" column here: http://aws.typepad.com/aws/2011/06/aws-lowers-its-pricing-ag...

oh ok, I thought "drops" meant get rid of but reducing is also great

Unlimited never is. I mean, there /is/ such thing as "don't worry about it as long as it's within an order of magnitude or so of what I expect" pricing, and that can often make a lot of sense on the low end or other places where the cost of the customer worrying about the bandwidth used is greater than the cost of providing that bandwidth... but this is not true of the 'at scale' uses of the cloud.

Outbound looks like it's dropped a few cents too although Asia's still expensive.

I wonder how soon/if tarsnap will update their pricing.

It might happen. I need to crunch some numbers.

It won't be overnight, though.

Reply on news.ycombinator.com