Distribute Your Content With Amazon CloudFront | Hacker News Reader

Distribute Your Content With Amazon CloudFront

aws.typepad.com

54 pointsjeffbarr38 comments

I'm seeing 38% faster downloads from Phoenix (nearest CloudFront edge: L.A.). I ran 5 trials on a 178MB video hosted on S3 [1] and CloudFront [2]. The S3 GETs took: 6m02s, 1m50s, -failed after 28MB- , 2m01s, 2m04s. The CloudFront GETs took: 3m40s, 1m10s, 1m23s, 2m05s, 1m25s. Ignoring the failed S3 run, we can compare average client latency: 2m59s vs 1m50s.

[1] http://2008.s3.amazonaws.com/09/javascript.mp4

[2] http://d11tgpuxrsgxu4.cloudfront.net/09/javascript.mp4

Surprisingly CloudFront was almost twice as slow as S3 for me (though I only tried it once):

[1] 6m 56s

[2] 11m 22s

Yes that is interesting without being shocking. True we can't conclude anything from one spin so if you have time please run it again and share. If it was an anomaly then I think it's: a) merely verification that the pipes downstream are unreliable and/or b) your edge location didn't have the file cached so fetched from S3 before serving to you. But if you continue to see unexpected results do tell.

S3: 10m 53s, 6m 24s, 10m 15s

CloudFront: 8m 37s, 8m 12s, 8m 17s

I wonder how this compares to Mosso's CloudFiles? They've already been offering a CDN built-in, but I haven't done any testing myself yet to compare their services (latency, etc).

Cost comparisons run very close using their cost calculators for just CloudFiles vs S3...

Mosso's Cloud Files uses Limelight Networks' system. It's not a one-off with Limelight either, it's the same CDN service any of their customers use.

In addition to ReST, PHP, Python, Java, and C# API's, there is also a web-interface that makes it easy to sign-up, upload content, and CDN-enable it.

Are there any guarantees on how fast files are invalidated on the edge servers? Running curl on those urls gives a lot of insight ( X-Cache: MISS ... ) into what's going on but it seems that invalidation of resources take 10-15 seconds. The X-Cache changes immediately but you only get a miss after a while. At least that's what I've been seeing in my pretty trivial tests.

This is general CDN question: how does one enable compression for CDN-served resources?

In Apache I just enable mod_deflate or mod_gzip, or serve up a .gz and set the Content-Encoding to gzip.

Do CDNs offer this somehow?

Can I set arbitrary headers like Content-Encoding?

Also, does anyone have stats on which browsers support deflate and gzip encodings? I imagine the vast majority at this point do.

the point of a cdn is to give you an SLA and honor it. how they configure their delivery platform should be opaque to you, its not like you are going to be getting a shell on the cdn machines anyway

So can someone else answer my question then? If I want my content gzip encoded are CDNs simply not an option?

With Amazon S3, you can set the Content-Encoding header to enable gzip. Here's a tutorial: http://devblog.famundo.com/articles/2007/03/02/serving-compr...

I'm not sure yet whether you can do that with CloudFront.

Amazon is a bad-ass.

They are doing a fine job, yep.

Something I couldn't find is support for SSL. That's key for CDN services.

Is it? Most of the use cases I can think of don't require SSL at all.

FWIW, I always found dealing with Akamai and HTTPS a bear.

Well, one of the most common problems are the images at the basket step(s) on SSL with usually low cache hit [ratio]. Servers end up doing routing for static content and backend content, adding complication to the architecture and for content editors. If you can place the images on SSL on the CDN it is much easier and you need less content servers.

The details vary significantly on each scenario. But as a global service I think Amazon will eventually have to add SSL, IMHO.

Signing up for a SSL certificate using your own credentials and then letting a third-party distribute and use your private key on an arbitrary number of systems you have no control over is bordering on fraud.

We are talking about a cloud service and the same company's CDN system. They already would have said certificates running possibly in multiple hosts. How do you get to "bordering on fraud" from there?

FAIL (Because of your reply tone, please try to be more civilized next time, shame on your up-voters too.)

If I give my certificate that says "this is Brian Smith" to Amazon, and then Amazon uses it to claim to be me, then that is very misleading. It is very similar to me giving my driver's license to my underage brother so he can buy me beer.

If CloudShare used its own "this is Amazon.com's Cloudshare Service" certificate then it might be okay, except (1) they would be using it to sign content that they haven't verified the authenticity of, which is misleading (like notarizing something without any authentication), and (2) you almost always need to mix those static assets with dynamic assets or HTML forms and that all has to happen on the same domain--the domain that receives the POST requests, not the CDN's domain--in order for things to work reliably.

As for tone: the tone with which I wrote my reply is apparently not the same as the tone with which you read it.

No (x3)

FAIL (Because of your reply tone, please try to be more civilized next time, shame on your up-voters too.)

Not everyone is interested in making sure your feelings don't get hurt on the internet. The sooner you realize that, the better off you'll be.

Hold on, who says you are handing over the private key?

I would definitely like to use a CDN with SSL sites. I am happy for the domain to be amazon's (or cloudfront, whatever) signed with their cert, I just don't want the browser security to be broken by having non-SSL content on the page.

Is there something wrong with this?

Mixing objects signed by different certs works the same way as having non-SSL content on the page.

Love the fact that you guys added the service to the Service Health Dashboard immediately: http://www.transparentuptime.com/2008/11/amazon-launches-fes...

Just read the docs, it seems that when you put a file in the CDN they distribute it on all edge servers and you paid for that (which is normal). But it would be great if you can chose on which servers you want the files distributed to. For example I might need the servers just from the european region.

If you have couple of files in the CDN it will not make sense, because the cost to distribute the files around will be marginal, but if you have many GB of data, distributing them to all unneeded servers might be not cost efficient.

CloudFront uses a pull model, not a push model. When a particular node needs the data and doesn't have it cached, the node fetches the data from S3.

Aha, 10x!

Can't wait to start using this for Stormpulse. Our traffic is extremely geo-specific (usually Texas and/or Florida), so this is going to help.

To be frank, I haven't been completely satisfied with S3, so maybe this will help insulate us from those outages as well, assuming the content is already cached at the edge.

Has anyone noticed that the transfer charges for the United States are actually less than the charges for S3 once you get over 10TB? Other than the added cost of distributing the files to the CDN, wouldn't that make the CDN as cheap or cheaper than S3? Great place to put your most accessed files.

That's kinda the point. Since bits from a CDN travel less distance, Amazon charges less.

OK, dumb question, but I thought there was this thing called Apache HTTPD that was traditionally used for serving static content? What ever happened to that? Granted, it has one basic flaw, in that it doesn't deliver usage statistics of my site to Amazon, but other than that?

Serving static content from Apache or lighttpd is trivial.

Putting multiple boxes hosting that httpd in various geolocations, setting up DNS and BGP to route users optimally, designing failover routes and keeping N nodes in sync is the hard part.

True, but when does one reach a size (user base) at which stuff like that becomes critical?

The same size one should start using any CDN - as soon as you start caring about page load speed.

OK, I forgot about that problem (I assume it is the limitations of browsers to two connections per server at a time).

You can work around that with multiple hostnames, but lower latency is still better.

And latency. And bandwidth.

from the pricing table, I noticed that the cost in Japan was the highest. I always thought that Japan had a lot of bandwidth.

Space + power? Also, AU and SE Asia will be routing to HK or Japan, and that would tend to be more expensive.

Reply on news.ycombinator.com