Aurora - New MySQL-Compatible Database Engine | Hacker News Reader

Aurora - New MySQL-Compatible Database Engine

aws.amazon.com

314 pointsak217131 comments

(Disclaimer: I work for an AWS competitor.)

To me, there is a very interesting contrast to be had between this announcement and Microsoft's announcement: it feels like Microsoft is discovering the business value of being open at the same time that Amazon is living in the time warp of proprietary everything. Has Microsoft internalized that open source is (or can be) a differentiator in the cloud? Amazon is clearly still oblivious to it -- and it will be very interesting to see if this service generates fear of vendor lock-in...

I have to agree with this. Amazon sees the competitive landscape and their move is to attempt to create some form of vendor lock-in by providing the services that their customers have asked for or are probably hosting themselves. This is by no means a bad move and will result in holding on to a lot more customers as well as creating new revenue streams from those services. For someone like myself however, it completely turns me off the AWS path. I will use AWS as a compute resource, as well as any other cloud provider. They will fight a battle for profit while compute costs continue to come down allowing me to reap the benefits while building a cloud agnostic stack.

Of course Amazon wants vendor lock-in. Their whole business model is about being a marketplace. Do you want them to open-source their shopping cart software?

And in Amazon Web Services, the biggest thing Amazon has to fear is the OpenStack initiative producing a platform that lots of little hosts can host.

At the same time, Amazon is stock market time bomb that will either start extracting monopoly rents and draw regulatory ire, or whose stock will drop, because of the amount of revenue multiples.

That said ... this is the way it nearly always works. Microsoft invented .NET over 10 years ago, and has kept it proprietary, leveraging the dominance of Windows, extracting rents. Now that Mobile is the new hotness and Microsoft needs to catch up, it goes open source. Google released Android for the same reason.

In short ... the companies that devote a lot of resources to R&D usually like to recoup the costs, but eventually PLATFORMS get opened. Frankly, the world benefits more when this happens sooner (Netscape open sources its codebase, the Web is designed, etc.) but the engine of progress in capitalism is still speculative investment, and that needs to pay off for years before the public at large can benefit, if investors are to continue investing.

Once we switch to crowdfunding, however, this model of capitalism and risktaking may be disrupted as well.

> And in Amazon Web Services, the biggest thing Amazon has to fear is the OpenStack initiative producing a platform that lots of little hosts can host.

I highly doubt OpenStack alone can make it possible for smaller players to replicate AWS's economics of scale, or even come particularly close. From what I understand Amazon doesn't just plop their software on commodity hardware, they use highly specialized gear optimized for running specific services. And that's not to mention buying power, the ability to run on razor-thin margins and other business factors that impact pricing.

that's almost all wrong. it's mostly commodity hardware, and their margins are thin only because they're a gigantic company. smaller companies can and do offer much better pricing, they just don't have the marketing budget to convince you of it.

> it's mostly commodity hardware

It's commodity in the sense that Amazon buys it from commodity vendors, but they use custom designs built to address the specific parameters of their data centers. Very few other companies can afford to employ an army of engineers to tweak every last component for optimal price-performance, nor for that matter do they have the benefit of accounting for a noticeable share of a supplier's revenue when entering the negotiations room.

> Smaller companies can and do offer much better pricing, they just don't have the marketing budget to convince you of it.

I am aware of at least one player (DigitalOcean) offering lower pricing than AWS, but that's only in a fairly limited niche. So while they may have the "economies" part checked they don't have the scale and scope to compete with Amazon on a particularly substantial level. I'd also be curious how they match up to Amazon in terms of operational efficiency since they lack many of the the advantages I listed here and in my previous post, which give AWS an edge in the race to zero as time passes.

The only reason that Amazon isn't immensely profitable right now is that their investors permit them to plow almost all of that profit into expanding their business.

In other words letting them create a monopoly and extract rents like standard oil?

How is this a case of creating vendor lock-in at all? The service itself is API compatible with MySQL... you can migrate to... MySQL, MariaDB, or a host of other services that are exposed in a MySQL compatible interface.

It is vendor lockin because it means you avoid thinking about how to deal with scaling beyond the single box.

So when the moment comes and you want to move away from AWS and you have scaled well beyond the capacity of a single box, what then? Suddenly you have to figure out sharding, replication and HA setups from scratch. While that may be the right choice for you if you're an early stage startup and it lets you defer engineering costs until a point where you're hopefully better funded, it's also dangerous:

It's far easier to get things like sharding and caching and use of read-replicas and multi-master replication right if you think about it from the beginning (if you've ever tried to retrofit any of this onto an application not designed with it in mind you know what I'm am talking about). It'd be awesome not to have to worry about this, but if you don't, you tie yourself into a platform you'll find it harder and harder to move off.

As a product, this solution would be a killer - it sounds fantastic. As an AWS-only service, I for one will treat it as if it doesn't exist, because getting tied to AWS for the long term is not an option for most of the projects I do.

There's nothing stopping you from using a scale-out architecture with Aurora, and nothing stopping you from running MySQL on your own hardware with loads of SSDs to get the throughput you are after.

Quote from article:

"Baseline storage performance is rapid, reliable and predictable—it scales linearly as you store more data, and allows you to burst to higher rates on occasion."

You mean to say that I can scale my single MySQL instance linearly with data storage by just throwing bigger SSDs at it? That has not been my experience, please share how you have been able to accomplish this?

Fair point, I don't know what voodoo they use, but you can purchase monstrously big servers (you'll need more than just one fit availability in any event) and storage arrays that would exceed the performance requirements of the vast majority of plausible use cases. The cost of such equipment is likely more expensive than AWS though.

You can also do sharding on mysql, or use foundation, or any number of other platforms that are mysql compatible as I said... lock-in means that you don't have the ability to migrate relatively easily. In this case, your application layer likely wouldn't need to change at all.

They've even mentioned it's pretty much a click-through to migrate from mysql... to me that seems like the opposite of lock-in.

As to only being able to use their infrastructure to scale this way, that's a value-add.. that's part of why you pay for SaaS instead of developing your own infrastructure. This is why you're renting instead of buying.

The interface is MySQL, no shortage of alternative implementations... Did Microsoft open source SQL Server?? How about the source code for ANY of their cloud services? They are ALL proprietary.

Amazon is way ahead of everyone else in the cloud space. Their closest rivals (Microsoft & Google) are pretty much copying AWS. Most of Azure's services are equally proprietary and I doubt that will ever change.

This just leaves the other smaller players (DigitalOcean, Joyent, Rackspace, etc) who are mostly offering something akin to EC2 and then partnering with other vendors to offer the missing pieces on top (frankly what other choice do they have? They can't get into a race with Amazon/Google on who can build the most no. of services - they will never win that race).

ignore OpenStack at your peril.

Why? Serious question.

OpenStack's model seems to be "clone what Amazon does, 18 months later then hand over to vendors who don't have the scale to match Amazon's prices"

It's kind of nice in theory for internal clouds.

But I'm increasingly seeing tooling targeting Docker instead of OpenStack at the service level for doing the same kinds of things OpenStack is supposed to offer (ie, the service utilities Docker to offer automatic deployment/availability/loadbalancing instead of doing it using the OpenStack APIs).

At the cloud vendor layer, the differences between vendors can be abstracted using libraries, which removes an alleged attraction of OpenStack.

Given that, I see a few vendors challenging Amazon by building unique selling points (Google has some innovative things as does Microsoft, and Digital Ocean pushes the price/performance thing).

I see Docker taking away a lot of the "cross cloud deployment" attraction that OpenStack had, and doing it better.

So what does OpenStack offer end users? (I understand it's attractions to vendors, and maybe the internal cloud use case).

Because renting bare metal servers and put your cloud on it is cheaper and better in a lot of cases.

Maybe -- the additional labor overhead needed to maintain OpenStack might negate the cost savings of running your own equipment.

When you are actually using it as a private cloud for multiple internal clients I can kind of see the point (as I mentioned in my previous comment).

If not, then what utility is OpenStack providing?

Bare metal works well. I see many people using either Docker on bare metal or maybe a VM layer as an additional security layer, but using Docker as the deployment target.

The utility it is providing is deployment/management of the underlying resources. Docker does not do that. Some of the stuff people are building on top of/around Docker will provide it, but it's not there.

But I agree with you that OpenStack is not good enough to be worth it for that with perhaps the exception of very large multi-tenant deployments (in which case, my first goal would be to start rewriting large chunks of it). And parts of it are just so horribly over-engineered it is scary, because it makes me wonder what it is they're missing to think it was necessary to make things that convoluted, and what they're missing because they've made it so convoluted.

I mildly degree with your assertion that the Docker tools aren't there yet. Some are pretty usable, and additionally if you include the category of DevOps tools then they are pretty good.

I agree entirely with you on the rest of your comment.

> And parts of it are just so horribly over-engineered it is scary, because it makes me wonder what it is they're missing to think it was necessary to make things that convoluted, and what they're missing because they've made it so convoluted.

So the source code for Amazon Web Services is clearly architected, free of cruft and under-/over-engineering? I assume you've worked with AWS's source code in order to be able to make such a comparison. In my experience, the design and architecture of closed-source internal applications delivered to customers only as a front-end is usually a nightmare.

I worked on and around Openstack for 18+ months. I think ignoring it is pretty safe to ignore at this point.

<rant>

There might come along something that is actually good, but Openstack isn't it. The architecture is just bad, it will never be as reliable as something like AWS. (not because of scale, purely because of lack of error handling capabilities)

Reality is these sorts of orchestration systems need to be written by specialists. The vast majority of Openstack was written by people that admit they have no clue about systems level programming. This is what hype does, it forces a whole bunch of bad programming and architecture down everyones throats.

The marketecture and hype machine have done horrible things to Openstack. Not only have vendors riddled the thing with lockin and crap code that needs to be supported, but they have pushed entire projects that should have been shot in the head. cough Ceilometer cough

There is security, performance and plain availability problems everywhere, mostly embedded deep in the architecture.

Dumb decisions like "All systems must be Python" when Python is clearly not suited to a large number of the things they want to do is painful. As is the general "not invented here" syndrome and boys club that leads to certain libraries or patterns being pushed over others, usually to the peril of the project due to the 'blessed' thing being incomplete and unproven.

</rant>

I don't intend on returning to Openstack if I can avoid it, unfortunately my skill-set does tend towards that sort of thing so we will see how I go.

So SmartDataCenter?

Why you said that when a lot of large enterprise are very actively investing in ? Can you tell more details ?

I've still to meet someone using it. Everyone uses aws; you hear people complaining about google compute's problems; I even hear people using eucalyptus. I've never met anyone using openstack (and I've been to aws conferences).

ever meet anyone who uses Rackspace Cloud? http://www.rackspace.com/cloud/

i agree with this comment completely. As an aws user, they have just done a great job in the cloud space. For anyone to really over take them, they'll have to come up with something radically different because the product is that good.

Apples and oranges. Cloud services are something very different from a development stack.

Amazon puts all its SDK stuff open source on Github, and I don't see Microsoft open sourcing its tech behind Azure.

Actually, the azure SDK bits are pretty much all on Github as well.. Azure's CLI interfaces are either via node.js modules (npm install -g azure) or, iirc powershell extensions.

I will agree on a lot of the tech.. I think, by comparison DocumentDB is horribly locked-in without an API-compatible version you can self-host (even if it doesn't use the same tech, or perform as well) is a big warning flag imo.

In this case, being MySQL compatible is anything but a lock-in.

It's 100% wire-compatible with MySQL 5.6 (using the InnoDB storage engine), so I'm not sure what you consider proprietary.

The fact that you can't run it outside of AWS?

Who cares? It's a service product, and AWS has many customers already who just want a faster RDBMS.

The point is you can run MySQL or any of its variants on your own hardware or anyone else's. Nobody forces you to use Amazon's implementation.

Once you have scaled to beyond the performance available on a standard setup on a single box without preparing to scale out, you face increasing costs in a migration because you potentially have to re-engineer your app to be able to run on a very different stack.

Compatibility at the connection level is just one part of a whole lot of issues to take into account when considering whether or not you can migrate elsewhere easily.

Given how expensive AWS is, that's something to seriously consider.

Well if only AWS provides the scaling you need then what would you migrate off of AWS to? You could re-arch your app to scale out for some use cases as you suggest, but that's also an indicator that using AWS allowed you to get to market faster with a much simpler system that didn't require a far more complex design, many more engineering hours and more admin.

Look more closely. MS may be opening the client side, but that is a honey trap to get people to use their server side stuff. And that is as proprietary as always.

It is the same old play book as they used for Exchange and IIS.

I really don't think that this is a 1:1 comparison by any stretch. The Amazon offering, by being MySQL compatible gives you a migration strategy.

By comparison, look at Azure's DocumentDB or Search services both of which obscure the interfaces to the underlying ElasticSearch ensuring lock-in.

Honest,y both have value, but this really isn't the same by any stretch.

Reading between the lines, it sounds like they've rewritten InnoDB's storage layer and made it multi-server aware w/ consensus.

I think the problem AWS is seeing is how they are being commoditized (e.g. you can just run your database on the cheapest hosting provider), so profits will move towards 0. They will need to add more services like this (Aurora, DynamoDB, etc.) to ensure AWS isn't a commodity (and you can't just easily switch to DigitalOcean).

I think that the AWS vendor lock-in won't come from single features such as databases from specific vendors, but rather the ecosystem of Route53, elastic load balancers, S3, EBS, AMIs, Glacier, RDS, CloudFormation templates, SQS, security groups, libraries like boto, auto scaling groups, and everything else besides instance provisioning. Of course you can implement most of these yourself, e.g., HAProxy for ELBs, rabbitmq for SQS, but all that requires application code changes and significantly more configuration management.

So far I've been able to avoid vendor lock-in on the code level (besides S3, but the S3 API is simple and has become pretty much a standard for object storage).

On the infrastructure level, the "lock-in" part is not so much the technology, all of that is relatively easy to replace. But it would take me two additional FTE's to configure and manage everything AWS offers as a service ourselves.

But that applies to a lot of things these days, AWS just happens to be a one stop shop for a growing range of commodity services.

> but all that requires application code changes and significantly more configuration management.

And not to talk about making them highly available, scalable, having to continuously monitor and so on takes time, effort and involves significant opportunity costs.

Imagine business folks telling engineers about an upcoming promotion campaign that bring in 3X more load. Outside of AWS (or cloud provided service) one has to sit and plan for scaling up all the individual components (rabbitmq, haproxy etc.,) which really becomes pain after a few iterations.

More product details: http://aws.amazon.com/rds/aurora/details

and Frequently Asked Questions: http://aws.amazon.com/rds/aurora/faqs/

Ouch. This pricing is pretty rough for SaaS sold on the premise of cloud scalability.

At $200/month for the entry level, their lowest price is many times what the cheapest geo-replicated "SQL engine as a service" from Google or Microsoft is. I'm not sure how the performance differs, but I am guessing theirs are no slouches.

Microsoft offers "SQL Database" geo-replicated for as low as $15/mo., and it scales up from there. Not sure about performance, but it would be apples to oranges (MySQL versus SQL Server) and difficult to compare. I wonder what the TPC numbers are, but apparently the TPC organization doesn't allow publishing that yet.

Google offers "Google Cloud SQL", also geo-replicated, and their cheapest pricing is between $10 and $18 dollars a month.

I don't think this is targeted towards those who only need to store a small amount of data and have very low performance needs. If you are are already spending $500/mo on an RDS instance, then it sounds like this would be a great solution. If you are spending $10/mo on a micro instance for your database server, I don't think this is targeted towards you.

The google cloud SQL pricing[1] for a comparable specs tier (D32) is 35$/day (~900+) which is way expensive compared to aws db.r3.large

[1] https://cloud.google.com/sql/pricing

It's not SQL Server, it's SQL Database. There are many large differences between the two.

A 50GB database with 10GB of RAM usage and 2 cores costs 700USD/Mo with SQL Database. This is replicated twice within DC. Double that price if you want georedundancy. Their entry level costs less but is also unusable for most real applications

Does this introduce more latency compared to running a LAMPs stack on EC2 or does it target larger databases? My database is about 300 megabytes.

You probably aren't the target audience if your database fits in RAM.

You can see notable changes in performance if your database gets fragmented, and tweaking the .my.cnf script seems like a massive distraction from my actual work. Having somebody handle these and improve performance for long queries would be wonderful.

With a database that small, performance shouldn't ever be noticeable. Are you running the DB on EBS?

1) If you are having trouble with tweaking my.cnf, give the perl script `mysqltuner` a go. It is pretty good.

2) The biggest improvement would be moving the storage to SSDs. I personally prefer Linode to DO, but both are cheap, and good.

My linode servers can do full text non-blocking backups at about 100MB per second, so depending on your outage recovery requirement, you may be able to get away with a simple cron job.

I you manage to have "long queries" with a 300MB dataset, you're doing something seriously wrong unless your definition of "long" is very different from mine. Or you're running on harware from the 90's.

Run "explain" on all your queries and review your indexes to begin with.

That dataset is so small that there should also be no need to be continuously tweaking the config file. 10 minutes looking at a performance guide, and ensuring you have enough ram to load everything into cache should be enough to get your configuration to a "good enough" shape - there's just nothing reasonable you'll be doing with a 300MB dataset that should need anything but getting the very basics of the config right to perform decently.

You can just use RDS m1.smalls then. We have databases that are 20-40GB running off smalls with relatively high throughput (more than I expected when builiding it for sure).

$200/mo minimum pricetag is extremely high for 300mb of data, or even 10GB of data.

We're looking at migrating a product of ours to Aurora and it has an operational dataset on the order of terabytes.

(inb4 nosql: Why not NoSQL? It needs transactions that don't suck (I'm looking at you Cassandra). We use C* for other large datasets, I do wish we could just use that.)

It must introduce a millisecond or two of latency b/c cross-AZ quorums aren't free. That will likely be trumped by whatever differences in implementation quality exist between mysql and aurora.

I have to agree with what others said. This is most likely targeted at the several GB + crowd.

Is a relational workload that requires four-nines-plus availability like Amazon is touting really something an organization would be willing to run in the cloud? Especially since that category includes a lot of personally identifiable data that legally can't be trusted to a third party. Wonder what kind of use cases they're aiming for.

people should be roughly as comfortable with this as with operating mysql on EC2 or using RDS, and I think that's quite common.

Amazon's environments meet PCI standards and there are folks with PCI Level 1 companies running their infrastructure in AWS and storing credit card data there.

“Aurora" is already the name of a cloud infrastructure management product, an open-souce apache project no less: http://aurora.incubator.apache.org/

Using an existing name for a product in a similar space is just confusing and hurts everyone.

To be fair, I just searched for "aurora software" and "aurora open source" and got nothing for that project on the first page of results.

A lot of the good names are already taken; there's going to be overlap, and they're very different products.

Name-clashing with an Apache project really surprises me. Also, consider the fact that Aurora stems from Mesos, which supposedly Amazon (cloud; although DB people are doing this one) folks have to know.

Michael Stonebraker created a database called Aurora more than 10 years ago.

http://cs.brown.edu/research/aurora/

Also, the first thing I thought of when I saw the word Aurora was the 2012 shooting in Colorado. That's sad that the word is associated with that, but also a facet of the naming consideration.

Why mysql? Why not postgres engine?

No one is saying that Aurora is built on MySQL, I'd personally like to know :) It's the protocol (drivers) compatibility what has been claimed so far.

I'd agree with you from a technical perspective it would have been better to offer Postgres compatibility, rather than MySQL. But I believe the intent is to target both a bigger market (as of today) and challenge Oracle.

> tables that I create in Amazon Aurora must use the InnoDB engine

It hasn't been claimed, but the article is filled with MySQL comparisons and references. I would not be surprised if it was a MySQL fork.

This is likely for the people who for whatever reason just aren't willing to switch to Postgres.

There are a lot of apps with complete ecosystems that aren't going to port to PostgreSQL anytime soon. Wordpress is the poster-child for this pattern. Yes, you can bring it up on PostgreSQL, and some modules support it, but from what I've heard, it's pretty spotty and most of the popular modules do not support it.

Even though PostgreSQL may be better. MySQL is still significantly more popular. That's why.

Postgres will probably follow. Redshift is basically Aurora for Postgres but tuned for data analysis.

4 times faster than MySQL on the same platform, how did they pull that off?

using SSDs looks like

Amazon's RDS MySQL already runs on SSD by default.

Yeah but we get hit by write/update contention constantly. :(

Would love to see if aurora fixes this for us.

SSD EBS most likely.

TokuDB claims to be 20x faster, so Amazon must be slacking. :-)

I don't vouch for any of these claims, but MySQL is certainly not the ultimate in DB performance.

Here is what they claim in the FAQ:

"Amazon Aurora delivers significant increases over MySQL performance by tightly integrating the database engine with an SSD-based virtualized storage layer purpose-built for database workloads, reducing writes to the storage system, minimizing lock contention and eliminating delays created by database process threads. Our tests with SysBench show that Amazon Aurora delivers over 500,000 SELECTs/sec and 100,000 updates/sec, five times higher than MySQL running the same benchmark on the same hardware."

I'd love to see more details (active benchmarking). At the very least, what is the CPU and disk utilization during the benchmark? It'll shed some light on how this was done: a 5x improvement in cycles per query? Or better caching?

People will benchmark this themselves ASAP. If you do, try to include some basic system metrics. The output of "vmstat 10", "iostat -x 10", and some "pidstat -t 1" would be a great start. This may only be possible for the MySQL benchmark, if Aurora is only visible via an API, and the database system can't be accessed directly (?).

To get the most comparable hardware setup for the comparison, I think you would want to run both Aurora and MySQL via RDS. RDS does not give you access to the underling EC2 instance, just to metrics via Cloudwatch.

Sounds to me like they are not using EBS like RDS-MySQL would be..

Has anyone been able to find specific pricing on this? All of the language I've seen about it has been vague. One of the things that's kept me from using RDS for toy projects is that it basically amounts to running another instance on top of what I already have. For a "real" project, that's a drop in the bucket, but for toys and prototypes, it can be a significant chunk of the cost (so I usually end up just running my own db on the same instance as the web server for those projects).

During the keynote they said that Aurora starts at $0.29 per hour (that's about $200 per month).

Pricing is at http://aws.amazon.com/rds/aurora/pricing/

The cheapest is a db.r3.large for $0.29 / hr

Ouch. Thanks.

Reserved pricing knocks that down a significant amount if you decide to stick with this.

Just slightly more expensive than the same instance with MySQL on RDS ($0.24)

Yes but "Storage is automatically replicated across three AWS Availability Zones (AZs) for durability and high availability, with two copies of the data in each Availability Zone."

Is that included in the server cost? Or will you actually be paying ~3x the server + storage costs?

Good question.

"Storage consumed by your database, up to 64 TB, is billed at $0.10 a GB per month and IOs consumed by your database are billed at $0.20 per million requests"

Sounds like the replication is included in this, but Amazon really should clarify that.

Based on how Amazon's pricing tables are, it should be included.

If you go to the normal RDS pricing, there's "Single AZ" and "Multi AZ" pricing. The Multi is 2x Single because it transparently runs the 2nd instance. http://aws.amazon.com/rds/pricing/

The pricing needs to be more clear. For example we run MYSQL RDS in production with two AZ enabled, we pay x2 the prices.... so if its only costs a bit more than MYSQL RDS and you get 3 AZ for the same price -- this is good. Moreover, there are costs that come with storage (database size), this thing seems to be smart about being more efficient with allocating space.

As i understood this your database storage is replicated to multiple places, but it still runs on one instance. It does not have failover to those replicated places. You'll have to launch multiple instances for that just like RDS. So cheaper, no.

Right, but you can also get much smaller MySQL instances. It's not necessarily an "ouch" given the capacity, but it is in combination with the fact that there are no smaller instance types.

If you need a smaller instance you are probably fine with regular MySQL instance though.

20% is "just slightly" more expensive?

For the claimed 5x performance and 6 way replication, it seems pretty cheap.

I'd think for most toy projects, running more than an EC2 instance or two is overkill. At least in my experiences, running the full stack on a single server works well enough for most hobby projects.

I certainly don't want to spend $7 a day playing around with RDS for fun!

Definitely. But one of the nice things about a lot of the AWS stuff is that you can build on the same infrastructure you'd use if you wanted to scale up. So if you have a toy project that ends up being more successful than you expect, it's pretty easy to just add resources. For example, DynamoDB is ultra cheap for the lowest usage levels, but simply by reserving more capacity you can scale up with absolutely no changes to your app. Makes it easy to prototype or play on the real system and not have to re-do your work later if the project becomes more than a toy.

I guess since this is MySQL-compatible it wouldn't be that hard to migrate from MySQL to Aurora, other than moving the data...

> other than moving the data

Moving data from one instance of MySQL to another is usually straightforward as a one-liner, piping a SQL dump from the old database into the new one over the network...

    mysqldump -hOldHost | mysql -hNewHost

...and waiting.

I'm interested in transaction support for e-commerce. Magento users can experience significant performance issues on MySQL, and the upper tier are always interested in more RDBMS performance, eg via NewSQL solutions.

So what can Aurora do for that workload? Do the support multi-table transactions and referential integrity across all 3 Availability Zones? Similarly, they mentioned Durability targets; what's their targets for Consistency (ie ACID).

This is beginning to feel like how it was when Intel was competing with AMD -- evolutionary improvements at first, which just kept on coming, increasing in size and impact, until AMD just faded out of the high-perf server market.

So, this is a Xen VPS (Amazon uses Xen as far as I remeber) with a MySQL 5.6 database, with a custom storage engine that stores data in some Dynamo-based storage. Any code could access it via MySQL protocol (bindings to libmysqlclient.so). Fine.

I appreciate your technical insight, but glueing those technologies together is far from trivial. Making it easily to manage and reliable is even harder.

I trust your best intention, but even Dropbox was dismissed on HN as user-friendly rsync.

OK, perhaps they wrote a MySQL-Dynamo (or whatever it is) wire-protocol translation engine, and do not use MySQL server. But custom storage engine looks like most natural way.

We will see it shortly.)

MariaDB enabled Cassandra as a storage engine: https://mariadb.com/kb/en/mariadb/documentation/storage-engi...

oh neat, AWS following GCP instead of the other way around.

What is GCP?

Google Cloud Platform.

In his defence even googling GCP doesn't return that as a result on the first page. I was searching for the same thing.

GCP is largely used as the acronym for "Good Clinical Practice" and has been around a heck of a lot longer than Google.

Most acronyms (especially TLAs) get overloaded daily. Being first means next to nothing, in this space.

care to explain?

CloudSQL on gce is mysql on bigtable afaicr, its been around for a few years.

EDIT: actually it looks like cloudsql might actually be a mysql.. also on gce they max out 16gb of ram and 100gb of storage

Aurora max looks like it's 64TB, that's 640 times larger, which I guess is what you're hinting at.

Your profile says you work at Google...

I certainly do. On cloud no less. The reality is that Amazon got to market several years before Google, so GCP had some catching up to do. That trend is changing, and this is one indication.

If you're going to post your subjective opinion on something involving your employer vs a competitor it's best to include a disclaimer in future.

What was the subjective opinion? "neat"? The rest was factual.

The subjective opinion was about "following" when Amazon has had relational database services for years, and the latest engine they say was under development for 3 years, not exactly something in response to goog.

Irrespective of that you shouldn't be posting as joe public when in fact you are a Google employee playing cheerleader for Google products on threads about your competitors products. Keep it classy.

We're not talking about a relational database, we're talking about a mysql-wire-compatible database. Google BigQuery has been around for a while too.

Adding "disclaimer: googler" to every post I make on HN seems pretty obnoxious to both me and anyone reading. I just didn't feel that "neat" + a fact qualified. Clearly opinions on that differ, and I'll probably just post less in the future.

It's only necessary in cases where there is an obvious conflict of interest.

That's fair. It's certainly not my intention to astroturf.

What is the use of a cloud relational database service? I thought everybody is going for Document Based databases.

If your data is fundamentally relational you're better off putting it into a relational DB. http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...

Just want to point out that the headline is sensational and there are many good use cases for document stores.

The headline doesn't mention document stores, it mentions MongoDB.

Which (as a document store) has plenty of use cases... and even when you read the article it's mostly about how they didn't understand what they were doing going into things... It's not a compelling argument.

"But this stuff wasn’t obvious at all. The MongoDB docs tell you what it’s good at, without emphasizing what it’s not good at. That’s natural. All projects do that. But as a result, it took us about six months, a lot of user complaints, and a lot of investigation to figure out that we were using MongoDB the wrong way."

The film/actor/career example they wanted to do could have been solved pretty easily with a document that had an _id for an actors collection and a name... that's just the document paradigm... I'm a huge fan of both postgres and mongodb and I know that's not a popular opinion around HN. I just get tired of seeing clickbait headlines being cited as the criticism within is correct.

I think the down votes to this are unfair, think you cor posting an enlightening response to someone who appears to be very ill informed :)

Not every data can be modeled as key value store, there are scenarios which require ACID transactions with relational data sets:-)

Reply on news.ycombinator.com