Hacker News Reader

Comment by __roland__ | Hacker News Reader

Sorry, I think that part of our write-up is misleading (I was involved in analyzing the issue described here). To our best understanding, what happens is the following:

- A client sends range requests and cancels them quickly.

- The full range request data will be billed (NOT the whole file), so I think this should read that the entire requested range gets billed, even if it never gets transferred (the explanation we received for this is that it's due to some internal buffering S3 is doing, and they do count this as egress).

In any case, if you send and cancel such requests quickly (which is easy enough, this was not even an adversarial situation, just a bug in some client API code) the egress cost is many times higher than your theoretical bandwidth (and about 80x higher than in the AWS documentation, hence the blogpost).

View on HN

This is a problem with lots of services. Blocking large quantities of legitimate looking requests is a hard problem. Request cancellation is also tricky and not supported well in a lot of frameworks/programming languages.

But the thing you are supposedly getting billed for is data transferred to the Internet. If the connection is closed there might be some data that goes out before the reset is received, but not that much. So either this is a bug, or the pricing documention is ... incorrect about what they actually charge you for.

Sadly this is no bug. Customers are NOT billed for actual data transferred. Customers are billed for "some kind of" data requested. If you interrupt the data transfer before downloading all requested data then it's "your fault". AWS documents this detail somewhere hidden on the S3 pricing page (https://aws.amazon.com/s3/pricing/). Search for "Data Transfer Out may be different from the data received" in the "Data transfer" tab.

That explanation still makes it sound like you are charged for what is transferred to the Internet, but your application might not get some of it because it already closed the connection.

From the OP it sounds like the amount billed is too high to be explained by that.

If what you are billed for isn't actually data transferred, then it is deceptive to say that you are billed for "data transferred out".

It like if the gaz compagny billed you for more gaz your pipe can physically take. No amount of fine print can change physics. If they bill for some opaque internal metric they need to change the wording to reflect this on all documentation: "we will bill you for whatever we please, you can't audit and we won't explain how it works".

Wait. What if the range is bigger than the file? Say if I made a request for a 5 TB range but the file is actually only 1kb. Would you get charged for 5 TB of egress?

This will return a range error instead of serving the file.

Reply on news.ycombinator.com