AWS’s anti-competitive move hidden in plain sight

In my opinion there are better examples, such as how AWS' managed products are deliberately hobbled and handicapped to make it complicated to migrate away from them once you're in. Two examples that I have first-hand experience with:

It's easy to set Aurora up as "slave" for accepting replication data from an external MariaDB/MySQL server in order to make a smooth low-downtime migration into AWS, but there are... oddly peculiar kludges... when wanting to set Aurora up as "master" and have it feed its binlog outside in order to migrate away in the same smooth fashion.

For their managed Redis ("Elasticache") they have intentionally disabled a few functions that relate to exporting/migrating live data from one Redis instance to another. In particular the "MIGRATE" function, which is used specifically to make a fast and accurate dump of all data when you want to supplant (read: migrate) a Redis instance. Instead you have to hack manual outbound migration together yourself, key by key, database by database.

At a former gig, the Prometheus server cost 10% of the total AWS bill in cross-AZ bandwidth charges alone. We were in a single region.

I have long thought it would be interesting to build a highly economical workload that is cross-region, but which only uses a single AZ in each. Inter-region bandwidth costs a little more, but if all your heavy traffic (monitoring etc.) is local you will save a ton. And since your only failover is cross-region, it should be easier to maintain the ability to do so as an organization. Also no extra exposure to whole-region failures :)

Ummm...

Quick back of the envelope [1] shows that the EC2 pricing is still only about 60% of the Aurora price, and something less than half of the RDS/PostgreSQL price, even counting the intra-region transfer.

There may be some transfer patterns and database sizes that cause the Aurora implementation to end up less expensive, but I think under most use cases Aurora is _more_ expensive, and RDS _much more_ expensive.

Just because the intra-region transfer isn't itemized doesn't mean that you aren't paying for it. Heck, the fact that the mark-up is so _low_ is pretty impressive, to be honest.

[1] Assumptions: [db.]r6g.large, x3 for ec2 instances, 1 year reserved capacity with monthly payments, 100GB storage, 1TB of intra-region transfer/month.

The author makes some glaring assumptions that these managed services are just sort of managed EC2 with no further optimizations. I find it very hard to believe that this is the case. Just on networking (which is his focus here) there’s a ton of opportunity to optimize networking between physical locations if you’re running a managed database service. It’s very reasonable for AWS to pass those savings onto their customers.

Corey likes to bash AWS for sport. Sometimes he has a point and other times he’s off the mark. Here he’s missed IMHO. He’s swatting in the air trying to find a controversy where there isn’t one.

> Let’s say I want to run something open-source in my AWS account; call it MySQL for this thought experiment. I can set up MySQL on an EC2 instance, or I can use AWS’s own managed service (in this case, RDS) to do it for me. I’ll pay slightly more for RDS, but that’s fair; there’s value in having AWS’s operational expertise applied to running infrastructure for me.

The author acknowledges they know they are already paying a premium for the managed service. With that premium you are also paying for the traffic between instances. If the price was the same I'd see how this is anti-competitive but like this I don't understand the argument.

It's fairly easy for them to defend though. They can offer "free" data transfer on products they control for good reason. Meaning they can tweak how efficient the synchronization is, what network paths it takes, predict when they need to scale it, etc.

Offering the same cost for customer managed tech would mean they take on risk where they have fewer controls and less visibility to manage and mitigate it.

Who are these engineering suckers who are spending all of this dollar-labor on solutions YOU KNOW full well at every opportunity will they want to find a way to lock you in, or to obscure engineering costs to you to make a deal look cheap, and then not knowing that leaving the lock-in costs more than writing a cloud independent solution to begin with?

I blame this lazy attitude of not knowing how to use your tools and instead asking cloud providers to know how to do things for you. Every single time I see an engineering team fail to understand how to use their tools and do their homework, they take the easy and long-term expensive way out. Oh and by the way, those solutions engineers you talk to barely know any better either. You will STILL eventually have to learn how to actually use the infrastructure you chose.

I write solutions for enterprises that are entirely independent of cloud specific offerings and then make sales engineers fight for volume business knowing they have no leverage.

Oh, you price hiked on us without notice? We already had deployment scripts to move providers and perform db redirections same-day and had already gone through procurement with other providers for insurance. Oh darn. Guess you shouldn't screw around with your customers.

As soon as you enter the pool, they WILL pull the ladder from you.

I correctly guessed this would have to do with bandwidth pricing, but I didn't successfully anticipate which specific aspect of bandwidth pricing; this quirk had never occurred to me before, but yeah... that sucks :(.

Intra-region traffic (i.e. inter-AZ) being free would make way more sense to a customer.

I’d really love to know what the marginal cost to AWS for this is though. I reckon there’s a very healthy margin in that 2c/GB.

This is not an "anti-competitive mode"

This is called a bundle

On the third-party side, we have: Compute + aws compute margin + network + aws network margin + storage + aws storage margin + software management + third-party margin

AWS is able to lower each margin parts, because they know you take a bundle

Why would you even think doing a cheaper but overall equivalent service in someone else's infrastructure .. THAT does not make any sense.

I’m completely not getting why any of this is anticompetitive. They offer a discounted rate to run their own services but their own services lack certain features and flexibility you would get if you run them yourself. That seems the very definition of the traditional IT infrastructure tradeoff - cost vs functionality. Having options in that space is good.

TFA basically says it’s anticompetitive because if you don’t use managed services you pay fees for interzone network traffic. Yes, but if you don’t use managed services you don’t pay for them so you save on that cost. Which cost tradeoff is best for you is going to depend on a lot of factors (how much interzone traffic, your use case etc) and you’re going to need to make a sensible decision that works for you.

In general I’m not sympathetic with the ethos of complaining about cloud service providers. They are great for many things but not all things and there are many other hosting options you can explore that may be better depending on what you’re trying to do. In many case it’s a cost vs convenience or cost vs functionality type tradeoff.

I don't see how this is anti-competitive. It's a crappy business practice designed to promote lock-in, but Google Compute and Azure both exist and are run by large, well-known companies that have the marketing and technical prowess to compete with AWS (and they do, even if not as effectively as they--or the consumer--would like).

You’re still paying for transfer, it’s just not itemized like hand rolled solutions.

The article should cover this, so maybe the author just isn’t knowledgeable enough to realize.

It’s like complaining that a buffet doesn’t charge for dish washing fees.

This is just an extension of what the egress fees, which are also anti-competitive.

Want to start a company called Snowflake, or Crunchy, or any of a number or similar providers? You’d better host in AWS (and Google Cloud, and Azure), because the respective cloud users wouldn’t want to pay for egress to use your service hosted elsewhere. Similarly, even Google can’t usefully undercut AWS’s pricing to get these providers to move out of AWS, because AWS would still charge the users.

I'm up for AWS bashing more often than not, but I don't get what this post is proposing. Should AWS overcharge for the RDS traffic that is handled differently than generic customer connections? Or should they do deep packet inspection on all traffic to pick out MySQL replication streams and bill them at lower rate because... reasons? Or something else?

What does the better version of this situation look like?

I believe locking customers in to proprietary services is called "going full IBM."

The argument would carry more weight if it included data on pricing; how much more do AWS managed services cost vs bare ec2, and what are typical amounts of bandwidth costs for the non-aws managed options. Just because AWS rolls the pricing under single pricetag doesn't automatically make it anti-competitive

How many services were saved by HA? The cost of HA is loss of performance and increased complexity in addition to monetary cost. If your application is not highly critical, I am not sure you are benefiting from HA. Is hackernews HA? nope

> I’ll have to pay 2¢ per GB that I replicate between AZs, whereas I will pay nothing if I use RDS.

This is surprising as I remember that data transfer on EC2 within the same region was free. Maybe I always got it wrong.

Overall egress/the bandwidth fee structure is probably more an anticompetitive move hidden in plain sight...

I've been directly involved in multiple systems being deployed on AWS solely to avoid egress fees

> I’m also not saying that this is some kind of mustache-twirling conspiracy on behalf of AWS to advantage their own services; I suspect this arose organically over time.

and so what if it was? all of what's described in the article are services offered by a company. It's hard to call that anticompetitive when it's their network and their services. It's like you're saying Toyota's car building divisions are anticompetitive because they don't allow Honda to build their engines.

If you don't like how AWS's pricing works you HAVE to vote with your wallet and support another vendor in their efforts.

As far as a list of moves by AWS that seem fishy, this one really isnt. This to me is closer to artificial scarcity pricing that matches a customers need and willingness to pay. Its no different than putting SAML in the “enterprise” column of SaaS pricing.

AWS had way more advantages and while this price is much higher than what the costs are, it’s sort of run of the mill business.

another victim of the ridiculous prices for egress traffic that most vps providers impose.

I have a suspicion that cloud vendors are using bandwidth pricing as a way to deter abuse. Can't use them for ddos, tunneling, file sharing or mass scraping that easily anymore. The lock-in is a nice side effect.

Though Corey is talking about internal network use and I am talking about bandwidth to internet

I wonder how much of this is a primary goal or just a nice side-effect for AWS. I imagine the custom internal forks of all this stuff, and their own data management systems are simply not compatible with the native variations that the products themselves have. From AWS's perspective it might seem wasteful to pro-actively support outbound data, and as a nice side-effect it makes it harder to leave or host an alternative.

For us, this is just part of the calculation; if we have special cases we run it ourselves, but the case also has to justify the inter-AZ replication and other side-effects.

The way we use AWS services also assumes migration paths outside of what you would normally do (binlog or CDC options) where we don't migrate 'the database' but 'the service' that relies on the persistence offering. This is of course much harder if some of your functionality relies on AWS-specific things, but for the average persistence system (ES, RDBMS, NoSQL, Redis, Memcache, object storage, Kafka, MQ) it really isn't all that exciting and the AZ cost is hardly a significant factor of the showstopper kind.

Perhaps this also is due to the way we designed our persistence systems; we only allow single-owner persistence, so no shared redises, databases or tables or schemas. We also don't do any binlog PITR, we use Kafka rewinds in between RDBMS snapshots since the database isn't the only factor in our consistency and partition tolerance conundrum. And Kafka itself is so 'thin' on the server side that binary exports/imports are only tied to the major version and highly dependant on what the client does (which is something AWS does not control - if they would their MSK offering would be useless).

Using AWS as a 'virtual datacenter' or 'managed version of the thing you already have' is bound to turn into a PITA. That applies to GCP as well, but almost 10x for Azure. Ironically, you're better off not using the big three if that is what you want, you don't get a lot of the features, but you also pay a fraction of the cost. And if you're not using the features anyway (or see them as a problem), Linode, Vultr, DO, even Scaleway are all a much better fit anyway.

In my cases where we use AWS, it's almost always for the same reasons:

  - We use enough of it to get discounts
  - We use enough of it to allow us to do 10x with the same people, it is a very significant force multiplier
  - We require some of the cross-service facilities like IAM, KMS, ACM, Security Groups

In a small number of cases it's also for compliance reasons, but I try to stay away from that since that usually comes with a checkbox clipboard manager/auditor regime as well, and I don't enjoy managing or working for those.

TLDR “Aws networking costs are a racket”

If I understand correctly, the author is upset that a business offers various pricing models to drive consumers towards certain decisions?

Doesn’t like, every company do this?