This is, oddly enough, similar to a debate people have about consumers TV or Internet: should pricing be "unlimited" or "a la carte"?
AWS is combining all your networking charges into one lump "outgoing data transfer" fee. So it's heavily marked up in comparison to what they're paying for the outgoing data transfer, and you're not sure how much is profit vs. whether it's going to cover all their other costs.
So it might be fairer if AWS broke out separate line items for internal, incoming and outgoing data transfer, plus all the additional systems a customer uses.
I think AWS's billing is probably already on the falling side of diminishing marginal returns. That is, it's complex enough that more information would tend to hinder customers from getting the best price. Right now, if I plan to reduce my data charges, I have one variable to tinker with. If we expand this, it would mean I'm having to balance incoming / internal and outgoing charges. That sounds simple, but in terms of engineering it can be very complex.
The next claim is that this biases customers not to move. Of course, Azure and GCP have the same arrangement, so while you pay to move out of AWS, you don't pay to move in to Azure or GCP. So all the vendors are attempting to lock you in to their product, and at the same time trying to extricate you from their competitors, overall it's a wash.
So, yes, part of the motivation for egress charges is that ingress is a loss leader. But it's also true that egress is a metric that does, for the vast majority of their customers, directly translate into customer value. If there's a compelling case for doing it differently, someone should do it and see if it works.
Cloudflare doesn't charge for bandwidth. I always throw cloudflare on top of anything I do, not because I really need a CDN or anything, but because the bandwidth cost would bankrupt me otherwise. The ceo of cloudflare gave the rationale on why they don't charge:
> There’s a fixed cost of setting up those peering arrangements, but, once in place, there’s no incremental cost. That’s why we have similar agreements to Backblaze in place with Google, Microsoft, IBM, Digital Ocean, etc. It’s pretty shameful, actually, that AWS has so far refused. When using Cloudflare, they don’t pay for the bandwidth, and we don’t pay for the Bandwidth, so why are customers paying for the bandwidth. Amazon pretends to be customer-focused. This is a clear example where they’re not.
They also do charge for Enterprise plans, but instead of transparent pricing I got high-pressure sales techniques and black box pricing offers - which then anchored our rate so that as we grow past our current contract, we're forced to upgrade at any point with pricing based solely on our original negotiation.
Frankly, while I save money using Cloudflare over Azure's CDN right now, it's left a very sour taste in my mouth and I'll be jumping their ship as soon as I have time to find a suitable alternative.
If you have the ability to shift your entire enterprise CDN away from them, why not first try renegotiating?
No, it's two variables - the egress charges you refer to and the actual cost to store the data.
We[1] have found that it is, as you might expect, quite a bit simpler to charge for just the storage and forget about metering the usage/bandwidth/transfer.
So we have typically had our price point higher than the B2s or Wasabis of the world, but there's just one simple number to think about - and no potential for surprises in the billing.
I will admit to having a bit of concern over adding 'rclone'[2] to our platform and the potential for users to just burn bandwidth using an rsync.net account as a "transfer host" but that is why we peer with he.net and their cheap an plentiful 10gb pipes.
[1] rsync.net
[2] ssh user@rsync.net rclone s3:/bucket gdrive:/blah/blah
Which is to say, each of our five[1] regional POPs have a single connection provided through a dumb switch one hop from he.net[2].
They have no interconnection or dependencies to one another.
No routers, no firewalls, no balancing, no failover. When rsync.net fails, it's a very, very boring failure.
We've had zero network outages in the last 60 months or so.
[1] Fremont, San Diego, Denver, Zurich, Hong Kong
[2] init7 in Zurich ...
The example given above for comparison, Hetzner, also doesn't charge for inbound and internal transfer AFAIK. Nor "the additional systems a customer uses". You pay a charge for the server, you get some amount of traffic included, and if you go over, the additional traffic costs something like $1.1/TB. That's all you pay.
> So it might be fairer if AWS broke out separate line items for internal, incoming and outgoing data transfer
This explanation doesn't cut it for me - most (all?) "traditional" VPS providers don't charge for ingress traffic, and I doubt anyone, ever, has charged for internal traffic.
So what exactly is 'all the networking charges' comprised of, other than egress data?
That companies don't charge for specific things doesn't mean those things don't cost them anything. It just means they're trying to work out a pricing scheme that scales with customer usage and is broadly understandable. So "data egress" is really just a proxy for "how much stuff you're doing with the networking subsystems of AWS."
Same thing with EC2, there are a whole pile of costs that are summed up with "time you rented an instance."
Of course there are is an internal cost of doing business, and peripheral infrastructure cost - but if I pay $100 for service "A" I reasonably expect that fee pays for service "A". Instead, egress bandwidth costs seem to be used to trick customers into thinking services are cheaper than they really are.
For egress bandwidth costs, I'd assume it included, well, the egress bandwidth cost.
But I think level3 is associated with:
https://www.centurylink.com/business/hybrid-it-cloud/public-...
And while they have a call-us price list (if you have to ask...) - they at least state:
"Public and Private high-capacity networking options up to 10Gbps. Note: there is no charge for internal data center traffic. Cost on a per-GB-out model"
I have no idea what they charge pr gb for this cloud product however.
And I think the charges for inter-AZ transfer are to incentivize customers to do that.
Of course, to make them fully independent, you have to replicate everything, so you wind up buying several redundant copies of your system...
Yeah, and keeping around warm systems ready to failover in case of a zonal outage seems like a preposterous waste of resources.
The alternative... to keep around multiple replicas of your system in different zones, all ready to accept traffic and which do serve traffic, seems more practical and less wasteful.
If instead of availability being the only value, there would be a more value provided from actually using such resources, more folks would adopt cross-AZ architectures which would be a win-win for both the customer (get HA for lower or no cost and go down less often and succeed in the market) and thus the cloud provider (keep raking in the steady cloud revenue as the customer grows).
This is one of those gotcha's that company's hit. They see the public pricing page and think "wow that is much cheaper than one my internal IT department charges for X", and then when they go to actually implement they find that "best practice" says they basically have to more than double or even triple the cost to get a reliable system (more because not only do you have to duplicate all the infrastructure into a second AZ, you are getting charged for the replication traffic between them).
This explains the cost.
Price probably should be based on value, not on cost. Why do they charge for it? Because they decided it's a good way to make money and profit.
That's pretty much it - in my experience most of AWS' awesome features are designed to lock customers into an environment where Amazon increasingly provides all of the components and services you need to do business.
I would assume the endgame is to create an ecosystem where the vast majority of customers allow their IT function (infrastructure, developing software, engaging third-party SaaS vendors, etc) to atrophy entirely, after which they'll have no choice but to buy what Amazon is selling, at any price, in perpetuity.
You could do this even just pegging the line 9 minutes per day and otherwise leaving it unused. ;)
People forget just how affordable it can be to maintain your own infrastructure. You can have the hardware and network capable of supporting 10X your average traffic loads and still have it operate far more cost effectively than the equivalent traffic on AWS.
At my work, we slashed our overall hosting costs by moving a data warehouse off of AWS and on to our own self-maintained infrastructure.
But with a bloated inefficient IT department or non-savvy negotiations with hardware vendors or transit providers, it can also be more expensive than AWS.
What you get from AWS is the logistics pipeline is already built as is the infrastructure should you suddenly require to serve factors of traffic more.
The ROI of AWS comes from the backend and capex vs opex debates.
Example : The CFO can go to the board and explain we are getting ready to reduce opex by laying off 10 developers at 150k yr during any meeting. The capex cost is usually fixed and hard to explain away.
The other is stupendous burst activity, like you just need a thousand(s) cores for a couple hours. Of course this doesen't mean the baseload has to be in AWS, just easy for small teams.
It's generally a function of network cost versus infra cost, and so the 'equation' solves differently based on the longest tech cycles, from inception to maturity to skill pool to diminishing returns and then back again on some other mode — this really is a decade+ thing.
Some "always true" inherent advantages of one approach (e.g. availability for cloud, or resources for on-prems) would remain across cycles as permanent gains; disruption then occurs when some new approach (e.g. containerization) fundamentally upsets the order of costs.
It used to be OpEx was easy, CapEx was hard to get approved. As people took advantage and OpEx went through the roof people are getting alot more pushback on reducing OpEx.
Pushing back on OpEx in favor of CapEx might be seen as a more long-term strategy too. Basically rent vs purchase, even for deprecating assets like infosys, as I think the current DevOps trend (scaling in pure software, virtualization, etc) makes it easier than ever to squeeze every last FLOP on-premises.
Okay, like AWS
> and doing that is a service that it’s probably worth paying for.
Okay, but how many orders of magnitude?
That's an expensive disk drive.
They want you to keep all of your data in their cloud, do all your processing there using their services (because doing it elsewhere incurs expensive egress fees), and get paid handsomely for your need to actually serve the data to your customers.
This also makes a migration additionally expensive, because you would need to egress all your data (old logs, some fresh backups, etc) in a short while.
So it's basically a soft lock-in.
I think it's more than that because a lot of folks aren't aware of the costs until they need to or want to move and then they get hit with a massive bill.
So maybe they chalk it up to experience and pay the bill because they have no other choice or calculated it's still better in the long term.
To me that's a much difference scenario than walking into a store and happily buying a dozen eggs for anywhere between $1 and $1.50. In this case you know what you're getting into before you make the purchase and everyone around you (other customers and businesses) decided that's what eggs will sell for in the open market. With outgoing data fees, it's more like a "take it or leave it" price dictated by the provider while they already have your data and there's no price competition since they are the sole business with your data.
Whether or not it's the customer's fault for not doing enough research is debatable, but it certainly doesn't help that most providers make it pretty difficult to calculate costs.
The market isn't exclusively good, and has many failure modes. This is one of them.
Sure, it’s the market. Why? Is it a Giffen good? Is it a case of very poor visibility of services to consumers? ...? There is something interesting going on, let’s figure it out.
Europe is probably the most competitive and best market in this case, almost all other markets are dominated by monopolies. Yes, even significantly worse than the Telekom monopoly.
Look at transit in Singapore, Japan or Australia. Or even in Brasil. You’ll go bankrupt even from trying to deliver a single movie to customers.
Hurricane Electric has POPs in both Singapore and Australia. Transport between Singapore and Australia is $1.50 or less. Peering ports are $0.20 per Mbps.
If all you want to do is push movies at customers, there are plenty of dedicated server providers who will sell you bandwidth on the cheap in both countries.
Obviously YMMV if you want better routes or direct interconnects with local monopolies.
Isn't DigitalOcean a cloud provider ($0.01/GB)?. Isn't Oracle a cloud provider (first 10 TB free, $ 0.0085 after)? Isn't OVH a cloud provider (free bandwidth)?
One thing folks like about AWS - you can actually know what you will be paying and there is no fake / hidden limits.
That's my somewhat outdated experience wasting a TON of time on this idea ages ago.
Give them a warning if you're going to spike your bill that high, but there shouldn't be any fake/hidden limits.
When you care about packet level SLOs... yeah, you start to shop around.
People under pressure make short term optimizations at the expense of long term strategy. There's nothing more to it. AWS reduced compute and IT expenses in the next few fiscal years, so people jumped all in on it. Solve the problem now. Someone will fix it in the future.
Actually, it would probably just make some programmers even more intolerable.
I've always got full 1Gbps out of Hetzner, even across the ocean, and for periods of multiple days of transfers.
(Like on all machines, you need to set the right TCP settings for windows sizes to make it physically possible.)