Hacker News Reader

Comment by bcantrill | Hacker News Reader

(Disclaimer: I work for an AWS competitor.)

To me, there is a very interesting contrast to be had between this announcement and Microsoft's announcement: it feels like Microsoft is discovering the business value of being open at the same time that Amazon is living in the time warp of proprietary everything. Has Microsoft internalized that open source is (or can be) a differentiator in the cloud? Amazon is clearly still oblivious to it -- and it will be very interesting to see if this service generates fear of vendor lock-in...

View on HN

Amazon is way ahead of everyone else in the cloud space. Their closest rivals (Microsoft & Google) are pretty much copying AWS. Most of Azure's services are equally proprietary and I doubt that will ever change.

This just leaves the other smaller players (DigitalOcean, Joyent, Rackspace, etc) who are mostly offering something akin to EC2 and then partnering with other vendors to offer the missing pieces on top (frankly what other choice do they have? They can't get into a race with Amazon/Google on who can build the most no. of services - they will never win that race).

ignore OpenStack at your peril.

Why? Serious question.

OpenStack's model seems to be "clone what Amazon does, 18 months later then hand over to vendors who don't have the scale to match Amazon's prices"

It's kind of nice in theory for internal clouds.

But I'm increasingly seeing tooling targeting Docker instead of OpenStack at the service level for doing the same kinds of things OpenStack is supposed to offer (ie, the service utilities Docker to offer automatic deployment/availability/loadbalancing instead of doing it using the OpenStack APIs).

At the cloud vendor layer, the differences between vendors can be abstracted using libraries, which removes an alleged attraction of OpenStack.

Given that, I see a few vendors challenging Amazon by building unique selling points (Google has some innovative things as does Microsoft, and Digital Ocean pushes the price/performance thing).

I see Docker taking away a lot of the "cross cloud deployment" attraction that OpenStack had, and doing it better.

So what does OpenStack offer end users? (I understand it's attractions to vendors, and maybe the internal cloud use case).

Because renting bare metal servers and put your cloud on it is cheaper and better in a lot of cases.

Maybe -- the additional labor overhead needed to maintain OpenStack might negate the cost savings of running your own equipment.

When you are actually using it as a private cloud for multiple internal clients I can kind of see the point (as I mentioned in my previous comment).

If not, then what utility is OpenStack providing?

Bare metal works well. I see many people using either Docker on bare metal or maybe a VM layer as an additional security layer, but using Docker as the deployment target.

The utility it is providing is deployment/management of the underlying resources. Docker does not do that. Some of the stuff people are building on top of/around Docker will provide it, but it's not there.

But I agree with you that OpenStack is not good enough to be worth it for that with perhaps the exception of very large multi-tenant deployments (in which case, my first goal would be to start rewriting large chunks of it). And parts of it are just so horribly over-engineered it is scary, because it makes me wonder what it is they're missing to think it was necessary to make things that convoluted, and what they're missing because they've made it so convoluted.

I mildly degree with your assertion that the Docker tools aren't there yet. Some are pretty usable, and additionally if you include the category of DevOps tools then they are pretty good.

I agree entirely with you on the rest of your comment.

> And parts of it are just so horribly over-engineered it is scary, because it makes me wonder what it is they're missing to think it was necessary to make things that convoluted, and what they're missing because they've made it so convoluted.

So the source code for Amazon Web Services is clearly architected, free of cruft and under-/over-engineering? I assume you've worked with AWS's source code in order to be able to make such a comparison. In my experience, the design and architecture of closed-source internal applications delivered to customers only as a front-end is usually a nightmare.

I worked on and around Openstack for 18+ months. I think ignoring it is pretty safe to ignore at this point.

<rant>

There might come along something that is actually good, but Openstack isn't it. The architecture is just bad, it will never be as reliable as something like AWS. (not because of scale, purely because of lack of error handling capabilities)

Reality is these sorts of orchestration systems need to be written by specialists. The vast majority of Openstack was written by people that admit they have no clue about systems level programming. This is what hype does, it forces a whole bunch of bad programming and architecture down everyones throats.

The marketecture and hype machine have done horrible things to Openstack. Not only have vendors riddled the thing with lockin and crap code that needs to be supported, but they have pushed entire projects that should have been shot in the head. cough Ceilometer cough

There is security, performance and plain availability problems everywhere, mostly embedded deep in the architecture.

Dumb decisions like "All systems must be Python" when Python is clearly not suited to a large number of the things they want to do is painful. As is the general "not invented here" syndrome and boys club that leads to certain libraries or patterns being pushed over others, usually to the peril of the project due to the 'blessed' thing being incomplete and unproven.

</rant>

I don't intend on returning to Openstack if I can avoid it, unfortunately my skill-set does tend towards that sort of thing so we will see how I go.

So SmartDataCenter?

Why you said that when a lot of large enterprise are very actively investing in ? Can you tell more details ?

I've still to meet someone using it. Everyone uses aws; you hear people complaining about google compute's problems; I even hear people using eucalyptus. I've never met anyone using openstack (and I've been to aws conferences).

ever meet anyone who uses Rackspace Cloud? http://www.rackspace.com/cloud/

i agree with this comment completely. As an aws user, they have just done a great job in the cloud space. For anyone to really over take them, they'll have to come up with something radically different because the product is that good.

I have to agree with this. Amazon sees the competitive landscape and their move is to attempt to create some form of vendor lock-in by providing the services that their customers have asked for or are probably hosting themselves. This is by no means a bad move and will result in holding on to a lot more customers as well as creating new revenue streams from those services. For someone like myself however, it completely turns me off the AWS path. I will use AWS as a compute resource, as well as any other cloud provider. They will fight a battle for profit while compute costs continue to come down allowing me to reap the benefits while building a cloud agnostic stack.

Of course Amazon wants vendor lock-in. Their whole business model is about being a marketplace. Do you want them to open-source their shopping cart software?

And in Amazon Web Services, the biggest thing Amazon has to fear is the OpenStack initiative producing a platform that lots of little hosts can host.

At the same time, Amazon is stock market time bomb that will either start extracting monopoly rents and draw regulatory ire, or whose stock will drop, because of the amount of revenue multiples.

That said ... this is the way it nearly always works. Microsoft invented .NET over 10 years ago, and has kept it proprietary, leveraging the dominance of Windows, extracting rents. Now that Mobile is the new hotness and Microsoft needs to catch up, it goes open source. Google released Android for the same reason.

In short ... the companies that devote a lot of resources to R&D usually like to recoup the costs, but eventually PLATFORMS get opened. Frankly, the world benefits more when this happens sooner (Netscape open sources its codebase, the Web is designed, etc.) but the engine of progress in capitalism is still speculative investment, and that needs to pay off for years before the public at large can benefit, if investors are to continue investing.

Once we switch to crowdfunding, however, this model of capitalism and risktaking may be disrupted as well.

> And in Amazon Web Services, the biggest thing Amazon has to fear is the OpenStack initiative producing a platform that lots of little hosts can host.

I highly doubt OpenStack alone can make it possible for smaller players to replicate AWS's economics of scale, or even come particularly close. From what I understand Amazon doesn't just plop their software on commodity hardware, they use highly specialized gear optimized for running specific services. And that's not to mention buying power, the ability to run on razor-thin margins and other business factors that impact pricing.

that's almost all wrong. it's mostly commodity hardware, and their margins are thin only because they're a gigantic company. smaller companies can and do offer much better pricing, they just don't have the marketing budget to convince you of it.

> it's mostly commodity hardware

It's commodity in the sense that Amazon buys it from commodity vendors, but they use custom designs built to address the specific parameters of their data centers. Very few other companies can afford to employ an army of engineers to tweak every last component for optimal price-performance, nor for that matter do they have the benefit of accounting for a noticeable share of a supplier's revenue when entering the negotiations room.

> Smaller companies can and do offer much better pricing, they just don't have the marketing budget to convince you of it.

I am aware of at least one player (DigitalOcean) offering lower pricing than AWS, but that's only in a fairly limited niche. So while they may have the "economies" part checked they don't have the scale and scope to compete with Amazon on a particularly substantial level. I'd also be curious how they match up to Amazon in terms of operational efficiency since they lack many of the the advantages I listed here and in my previous post, which give AWS an edge in the race to zero as time passes.

The only reason that Amazon isn't immensely profitable right now is that their investors permit them to plow almost all of that profit into expanding their business.

In other words letting them create a monopoly and extract rents like standard oil?

How is this a case of creating vendor lock-in at all? The service itself is API compatible with MySQL... you can migrate to... MySQL, MariaDB, or a host of other services that are exposed in a MySQL compatible interface.

It is vendor lockin because it means you avoid thinking about how to deal with scaling beyond the single box.

So when the moment comes and you want to move away from AWS and you have scaled well beyond the capacity of a single box, what then? Suddenly you have to figure out sharding, replication and HA setups from scratch. While that may be the right choice for you if you're an early stage startup and it lets you defer engineering costs until a point where you're hopefully better funded, it's also dangerous:

It's far easier to get things like sharding and caching and use of read-replicas and multi-master replication right if you think about it from the beginning (if you've ever tried to retrofit any of this onto an application not designed with it in mind you know what I'm am talking about). It'd be awesome not to have to worry about this, but if you don't, you tie yourself into a platform you'll find it harder and harder to move off.

As a product, this solution would be a killer - it sounds fantastic. As an AWS-only service, I for one will treat it as if it doesn't exist, because getting tied to AWS for the long term is not an option for most of the projects I do.

There's nothing stopping you from using a scale-out architecture with Aurora, and nothing stopping you from running MySQL on your own hardware with loads of SSDs to get the throughput you are after.

Quote from article:

"Baseline storage performance is rapid, reliable and predictable—it scales linearly as you store more data, and allows you to burst to higher rates on occasion."

You mean to say that I can scale my single MySQL instance linearly with data storage by just throwing bigger SSDs at it? That has not been my experience, please share how you have been able to accomplish this?

Fair point, I don't know what voodoo they use, but you can purchase monstrously big servers (you'll need more than just one fit availability in any event) and storage arrays that would exceed the performance requirements of the vast majority of plausible use cases. The cost of such equipment is likely more expensive than AWS though.

You can also do sharding on mysql, or use foundation, or any number of other platforms that are mysql compatible as I said... lock-in means that you don't have the ability to migrate relatively easily. In this case, your application layer likely wouldn't need to change at all.

They've even mentioned it's pretty much a click-through to migrate from mysql... to me that seems like the opposite of lock-in.

As to only being able to use their infrastructure to scale this way, that's a value-add.. that's part of why you pay for SaaS instead of developing your own infrastructure. This is why you're renting instead of buying.

The interface is MySQL, no shortage of alternative implementations... Did Microsoft open source SQL Server?? How about the source code for ANY of their cloud services? They are ALL proprietary.

It's 100% wire-compatible with MySQL 5.6 (using the InnoDB storage engine), so I'm not sure what you consider proprietary.

The fact that you can't run it outside of AWS?

Who cares? It's a service product, and AWS has many customers already who just want a faster RDBMS.

The point is you can run MySQL or any of its variants on your own hardware or anyone else's. Nobody forces you to use Amazon's implementation.

Once you have scaled to beyond the performance available on a standard setup on a single box without preparing to scale out, you face increasing costs in a migration because you potentially have to re-engineer your app to be able to run on a very different stack.

Compatibility at the connection level is just one part of a whole lot of issues to take into account when considering whether or not you can migrate elsewhere easily.

Given how expensive AWS is, that's something to seriously consider.

Well if only AWS provides the scaling you need then what would you migrate off of AWS to? You could re-arch your app to scale out for some use cases as you suggest, but that's also an indicator that using AWS allowed you to get to market faster with a much simpler system that didn't require a far more complex design, many more engineering hours and more admin.

I really don't think that this is a 1:1 comparison by any stretch. The Amazon offering, by being MySQL compatible gives you a migration strategy.

By comparison, look at Azure's DocumentDB or Search services both of which obscure the interfaces to the underlying ElasticSearch ensuring lock-in.

Honest,y both have value, but this really isn't the same by any stretch.

Apples and oranges. Cloud services are something very different from a development stack.

Amazon puts all its SDK stuff open source on Github, and I don't see Microsoft open sourcing its tech behind Azure.

Actually, the azure SDK bits are pretty much all on Github as well.. Azure's CLI interfaces are either via node.js modules (npm install -g azure) or, iirc powershell extensions.

I will agree on a lot of the tech.. I think, by comparison DocumentDB is horribly locked-in without an API-compatible version you can self-host (even if it doesn't use the same tech, or perform as well) is a big warning flag imo.

In this case, being MySQL compatible is anything but a lock-in.

Look more closely. MS may be opening the client side, but that is a honey trap to get people to use their server side stuff. And that is as proprietary as always.

It is the same old play book as they used for Exchange and IIS.

Reply on news.ycombinator.com