MongoDB 3.4.0-rc3
jepsen.io
jepsen.io
Open and available for everyone to see, for every build of MongoDB. Is there another database that has this much transparency? (for every build)
Not exactly, but we published a blog post about why we took the time to write a new CI system, and that post does touch upon why we didn't use Jenkins + plugins.
https://engineering.mongodb.com/post/evergreen-continuous-in...
I at least will never trust Mongo for anything but a toy project. There are so many better options out there, options whose technical capabilities are as good as Mongo's marketing.
That depends on the data.
What type of data you have and what you want to do with it.
MongoDB isn't data specific and it claim to fame is flexible data structure.
If you want fast write and look up with very little relation Cassandra is good.
If you want searchable text document then anything that is base on Lucene is good (ES, Solr, Raven).
If you want time series there are few out there but it's a niche.
Likewise if you want graph data then there are NodeJS, Titan, etc..
MongoDB at most company I worked with was use because they don't think about what type of data it is and what performance they want. They want to store unstructure data cause it's easy.
I personally think it's a cop out, especially as a statistician/programmer.
And, I can't sell that to people I work with.
I think you meant Neo4j.
* I only want starts-with and contains on strings.
But with Jespen tests MongoDB can finally be considered a contender. Its not like competent teams were using it in production. Right?
Please, name some F500 companies using RethinkDB to power critical infrastructure. There are many using MongoDB. While Rethink is widely renowned among the HN set it is nonexistent in comparison when looking at actual deployments.
The reigning HN view of MongoDB being a buggy mess is outdated. Yes, they overmarketed a buggy project in 2009. It didn't matter, because they built a product that developers loved (and continue to love) to use. RethinkDB didn't aggressively market itself, and look where it is now - defunct. Mongo used that momentum to raise money and hire an incredible engineering team, including Keith Bostic, one of the fathers of Unix, and Michael Cahill, the inventor of the transaction isolation mechanism used in Postgres. Sometimes you need to employ aggressive business tactics to get to a point where you have the engineering resources to build a world class project. Moreso when you need to catch up to millions of man hours spent building Oracle and MSSQL.
I should note that I work for a multi-national gaming company and we use software that is ABSOLUTELY not fit for purpose, but once you have a hard dependency on something and the cost of muddling through is _less_ than the cost of a rewrite then you're going to be stuck supporting it.
This is the reality of technology in enterprise.
Looking at the numbers, larger organizations straight-forwardly seem like they should be more likely to eventually hire mediocre talent, survive despite having done so, and more likely to have adopted any given tool.
I specifically challenge that you can do so, given a model of a large organization as being necessarily more diverse (regressing to the mean of general competency, more likely to have facts like "org Y uses tech X" being true) and more robust to survive failures as it grows.
We don't have data, but we can still model (if nothing else, to think about what data we would need).
Mongo, on the other hand, loses data.
[1] https://www.indeed.com/jobs?q=mongodb&l= [2] https://www.indeed.com/jobs?q=rethinkdb&l=
Both started out being written by people who know nothing about databases and both threw away years of database research.
Both gained popularity due to being accepted choice by web-based languages (PHP vs NodeJS)
Both were faster than more established competition, only to turn out that both were losing data.
Both turned out to be designed fundamentally wrong and had a replacement engines that are more reliable (ISAM/MyISAM vs InnoDB and v0 vs v1).
Both still have quirks due to bad decisions in the past, but which can't be easilly fixed due to breaking compatibility.
One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in to the hands of Oracle. MongoDB was smart enough to make sure that they were the acquirer, which puts them in control of their own destiny.
If MongoDB is heading along the path of MySQL, that's a pretty good path to be on considering that MySQL is used as the store of record at Facebook, Twitter and some parts of Google.
My bad, MMAPv1 vs WiredTiger although I think it was obvious what I meant.
> One big difference from a corporate strategy perspective is that MySQL let the replacement storage engine (InnoDB) fall in to the hands of Oracle. MongoDB was smart enough to make sure that they were the acquirer, which puts them in control of their own destiny.
Not sure if that's relevant though, since whole MySQL became property of Oracle (after they acquired Sun).
I've heard that about 1000 times a day for 6 years. Usually the person stating it is snickering as if they are clued into some unknown secret.
Mongo does work in production at many shops, and in many forms. Sometimes it's used as the main database, sometimes it's used to house specific slices of data, etc.
MongoDB is the Nickelback of databases: a reasonable act that's not going to blow your socks off, but one where saying "OMG I hate it!" somehow signals membership to some cool clique of connoisseurs.
The main win was not server stability, it was having general tools to manage data .. including the inbuilt geo-algorithms that come with postGIS. eg. I could make our data set 9x smaller by smoothing map paths.
I still love the Mongo api, but I just cant risk it with data on projects that people are paying for, or that I need to support.
[ I think the real sweet spot will be deep integration of javascript and json into postgres - so I can write stored procs in js, get db events in js, wrangle json fluidly.. all of which is improving. ]
>OMG I hate it!" somehow signals membership to some cool clique of connoisseurs
You do realize you are the one dragging identity into the mix.
I dont understand how this is a big ego debate. MongoDB isnt that relevant and it isnt Nickelback, its a homeopathetic database. That is: if you use it for something else than caching (store data you cant afford to loose, or load balance and use the database as main mutex to deal with all concurrency issues) that would arguably be a very irresponsible choice.
Hell i hate most databases, because its hard to get right yet some have interesting trade-offs (ElasticSearch, Cassandra, CockroachDB).
And its not a subjective or even analog discussion where databases are more or less consistent or more or less durable. They fsync or they dont. They use raft with majority consensus or they dont.
If you as a sysadmin judge these emperical facts based on your prejudices about the sort of people that would agree or disagree with you than you are much more like the cool clique of connoisseurs than the people at the other end of your finger.
Its engineering, not wine tasting. The shape of the world isnt a subjective thing anymore than the durability of a database that doesnt fsync.
Maybe. But you'd have lost that bet if you made it any day in the last 10 years. And in those 10 years mongodb has demonstrated again and again that they aren't up to the task of writing a reliable database. Even with their new storage engine they couldn't find the bugs alone.
I think using mongo today for any mission critical data is an irresponsible choice. I'd seriously question the judgement of any senior engineer who picks it for a new project over rethinkdb or Postgres.
Given a small cluster of reliable nodes on a reliable network, these errors will occur extremely rarely. So rarely, in fact, that they'll be written off as "user error" by support.
If you're a startup building a system which has to quickly and reliably scale from 3 > 3000 nodes in a year then the whole thing is likely to explode in your face. Twitter style.
Now, if MongoDB was so superior that it was truly platform which would even enable that kind of scaling, then the decision is simple: just go for it.
The thing is, this isn't how the world works. When systems are built, very few people consider (or are capable of considering) the growth of the system. Frameworks and database are, by the rule, chosen arbitrarily. When scaling happens, the question is more "how can we scale what we have whilst having everything kind of work" than "how do we design a system which works correctly at scale".
Mongo's whole strategy is based around this. Make Mongo the default choice for the current generation of developers.
Fantastic market strategy.
When you talk about growing, the biggest value in Open Source has been that you can start with something free but shit, and then as you make money then you can spend it on customizing that Open Source in a way that benefits you.
However there exist commercial offerings that are (and were) faster and better at MongoDB than MongoDB was: KDB could've handled Twitter, we never would've seen a fail whale, and it is a whole hell of a lot cheaper than the developers and the customizers, and the headache, and the fact that you're making something open source which ultimately benefits your competition.
Another way to think about it is by thinking about experts: If you've got a great startup idea, why would you want to make your odds 10% worse by introducing the possibility it'll fail, by using the cheapest hacky hack thing that has 10% chance of losing your data? Ask experts with data, and be honest with your budget and you'll do a lot better.
How does KDB handle replication and failover? Or even high insert/update rates to datasets that exceed the size of memory? How do you shard KDB?
KDB doesn't support unicode text. Do you plan to only have English speaking users?
Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 10 TB or less) numerical time series data. It would be a horrible choice for the backing store of a high throughput CRUD application...
What is it with KDB zealots thinking that KDB is the best database for every task? I swear, KDB is the Scientology of databases.
Unicode (from 2011):
With -r and (in my case) SO_REUSEPORT. Most people use a dedicated gateway (have seen custom tomcat stuff and haproxy).
Meanwhile, MongoDB doesn't actually replicate reliably (acking then losing anyway) and failover can crash cascade in the naïve configuration.
> Or even high insert/update rates to datasets that exceed the size of memory?
This is literally the KDB tickerplant model. Have an RDB that flushes out regularly (daily) to an HDB.
You can also just write to a log `:log upsert ...
> How do you shard KDB?
Same way you shard anything else? By picking a key and directing the query to the appropriate server. h[(first md5 k) mod count h] "query..."
> KDB doesn't support unicode text.
UTF8 is fine.
The number of times I've needed the first 5 code points (and not the first 5 bytes or the first 5 characters) in my life is zero. All that half-baked Unicode support in various languages (like MongoDB) just makes people think that they've solved a problem that they really haven't.
> Yes, KDB excels at its relatively well defined niche of transforming and aggregating "smallish" (say 10 TB or less) numerical time series data. It would be a horrible choice for the backing store of a high throughput CRUD application...
I use it in one of those big CRUD databases (digital marketing and tele-lead tracking).
> What is it with KDB zealots thinking that KDB is the best database for every task? I swear, KDB is the Scientology of databases.
Because it solves problems they have.
Even when I don't use KDB I use a similar architecture because it's the correct architecture, because I've had these problems for a lot longer than I've had KDB.
If it doesn't solve every problem I have, that's because I have work to do, not because it isn't great at the problems it does solve, and I don't shout at my hammer because it isn't a spoon.
However MongoDB doesn't solve any problem I've ever had: I've never needed a bag of objects/filesytem that loses data, or a binary blob that I cannot query. It's so famously "web scalable" it has made a joke of the very idea of being scalable.
So as a KDB user you need to implement your own HA solution. That is strictly worse than MongoDB replication, even with its now-fixed bugs. Do you really think your homemade multi-master KDB system would pass Jepsen?
> This is literally the KDB tickerplant model. Have an RDB that flushes out regularly (daily) to an HDB.
Wat? That only works if data is immutable once written. Tweets are liked/deleted/etc. You could store an immutable log of user actions, but then you would have to reconstruct the current snapshot every time someone loads a timeline. It's entirely possible for someone to like/delete/RT an old tweet. Financial data is naturally partitioned because the order book clears at the end of every trading day - this doesn't apply to CRUD apps.
> UTF8 is fine
I think you misunderstand what I mean by unicode support. Does KDB support locale specific collations? Does it support normalization/canonicalization? Being able to index by code point is about 1% of the needed solution to build an i18n-proof product. Obviously that doesn't matter when you are dealing with normal KDB datasets like market data where e.g. asian names are represented with numbers.
> I use it in one of those big CRUD databases (digital marketing and tele-lead tracking).
Were you using it to store clickstream data? Or some other kind of immutable stream of events? That isn't really applicable to general CRUD applications.
Like I said - KDB is great for analyzing immutable streams of events. It's not a general purpose database for building CRUD applications. MongoDB tries to be a reasonable enough solution for many use cases, while KDB focuses on excelling at a small number. Both are valid approaches to building a database...
You simply cannot take MongoDB in its (near) default configuration, put it on AWS, and handle Twitter volumes.
I think "this broken tool is better than your working tool" represents a certain kind of madness that I can't argue with.
> Wat? That only works if data is immutable once written. Tweets are liked/deleted/etc. You could store an immutable log of user actions, but then you would have to reconstruct the current snapshot every time someone loads a timeline. It's entirely possible for someone to like/delete/RT an old tweet. Financial data is naturally partitioned because the order book clears at the end of every trading day - this doesn't apply to CRUD apps.
I don't know what your experience level is, but Financial data typically has many subscribers to that tickerplant, and build-up indexes an views representing the queries that consumers are actually going to be interested in. This is covered in the most basic of KDB tutorials[1]
When a user loads a timeline, ideally you want to hit a single machine on a single query to a machine near the viewing user. Processes representing tweet consumers subscribe to the ticker plant, and build-up the indexes of what information they're going to need to publish. You're also going to need to have a fast index-by-publisher as well, so that when a subscriber wants to follow someone, we don't need a replay -- again, more indexes, but at least these can be "centrally" located.
This isn't even a remotely difficult problem to solve with the right tools.
[1]: http://code.kx.com/wiki/Startingkdbplus/tick
> I think you misunderstand what I mean by unicode support. Does KDB support locale specific collations? Does it support normalization/canonicalization? Being able to index by code point is about 1% of the needed solution to build an i18n-proof product. Obviously that doesn't matter when you are dealing with normal KDB datasets like market data where e.g. asian names are represented with numbers.
If I misunderstand you, it is because you are unclear.
JavaScript, C and C++ don't actually support "locale specific collations" even though there are well-maintained and well-distributed collation and localisation libraries that people can use.
That "iasc" doesn't know the difference between Chinese and American spellings for a word is irrelevant. I can solve the problems I have with my tools, and building sort keys on my symbol tables for each locale means that the user-visible aspects of sorting remain instantaneous, instead of being tricked into doing stupid shit like x.toLocaleString(user.getLocale()) which is slow at Twitter scale.
This is part of what I mean by "engineered correctly": The tools that are available to us can trick us into thinking certain problems are solved when they aren't.
> Were you using it to store clickstream data? Or some other kind of immutable stream of events? That isn't really applicable to general CRUD applications.
Tele-lead means (outbound) phone calls for the purpose of lead-generation, so I have phone calls and the results of those calls in KDB. It's not "big data" by any stretch of the imagination.
> Like I said - KDB is great for analyzing immutable streams of events. It's not a general purpose database for building CRUD applications. MongoDB tries to be a reasonable enough solution for many use cases, while KDB focuses on excelling at a small number. Both are valid approaches to building a database...
MongoDB is not a valid approach full stop: Build dogshit and then try to pepper over the bad press with "the new version isn't dogshit anymore" every few years is negligent at best, and pays dividends with the fact that it makes it easy to identify inexperienced engineers.
That KDB is not as accessible as MongoDB is Kx's problem, and not KDB's problem.
You check the result with getLastError which, as you described, can be used to ensure a majority agrees with the write. But you normally don't use getLastError that way. Because a majority might not even be concerned with that particular write. They are, after all, shards. Instead you check if primary got the write. If primary disconnects while you are checking, you catch the exception and try checking until a new primary is decided. And if your check result is not ok, you try inserting again. That's as reliable as it gets when inserting to any database including SQL databases that support transactions.
>that's as reliable as it gets when inserting into any database including SQL
The difference being in a SQL database you call commit and all this happens for you automatically
ah, no. I did not.
I don't really understand the angle of "can I get away with it anyways, tho?"
> I'd seriously question the judgement of any senior engineer who picks it for a new project over rethinkdb or Postgres.
... you mean RethinkDB, whose future is still uncertain? Regardless of technical merits, the currently unstable future of RethinkDB means a senior engineer should be extremely cautious about selecting it for a significant project.
Didn't WiredTiger Inc write the new WiredTiger storage engine before they were acquired by MongoDB Inc?
https://gigaom.com/2014/12/16/mongodb-snaps-up-wiredtiger-as...
> As far as I can ascertain, RethinkDB’s safety claims are accurate. You can lose updates if you write with anything less than majority, and see assorted read anomalies with single or outdated reads, but majority/majority appears linearizable.
The safety claims are accurate but like all databases, its never 100% perfect as there are tradeoffs.
https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu...
> That reordering was only possible because of a special workaround for a bug in an older version of RethinkDB.
> What are the risks to users? The RethinkDB team and I suspect it’s unlikely this bug will occur outside of stress testing. Cluster reconfiguration is typically infrequent, and users would need a specific series of network failures or other message delays which happen to cut the replicas apart—in a way which allows both network components to find independent majorities for their respective table configurations. In Jepsen tests, it usually takes tens to hundreds of partition/reconfigure rounds to trigger this bug.
This isn't really an architectural failure (as you see repeatedly with MongoDB) but an implementation bug.
Compared to the failures in MongoDB that are frequently architectural (rather than implementation bugs), RethinkDB passed easily and quickly fixed the implementation bugs.
Depending on the exact requirements Postgres (in the right configuration) and Redis are better options.
I'd argue that for 90% of MongoDB users - the group whose requirements were met by MySQL/MyISAM - would be better served by Postgres or MySQL with a proper storage engine.
Wait, what?
As for the post on HN from a few months ago, I remember only one guy who mixed LWT and non LWT updates and was surprised with lack of linearizability. Not a Cassandra fault if somebody doesn't know what he's doing.
I suggest you re-read the analysis. Some databases can offer safe, generalized commutative updates; e.g. Riak. Cassandra can't: updates, in general, can be lost through reordering.
> There is no better way than last write wins if you want high availability and partition tolerance and don't want to pay the performance and availability price for a consensus algorithm like Paxos or Raft.
There is. There's a whole field of research devoted to this problem. http://hal.upmc.fr/inria-00555588/document
http://www.datastax.com/dev/blog/why-cassandra-doesnt-need-v...
As for the research you posted, there is no free lunch. Each of these strategies come with their own set of drawbacks. That's why Cassandra offers choice at a query level.
That's why Cassandra offers choice at a query level.
It doesn't give you a choice: you get last-write-wins, or some limited merge functions with CQL types. You can't get generalized CRDTs in Cassandra, because they won't expose conflicts to the user layer. Cassandra gives up on a whole class of safe AP datatypes as a consequence of this restriction.
They will, you'll see them as separate rows of the same partition, and then you can merge them as you wish in the application.
> While the v0 protocol remains broken, patches for v1 are available in MongoDB 3.2.12 and 3.4.0, and now pass the expanded Jepsen test suite.
Given the projects I chose to work on, MongoDB would be a bad choice. So I would only use it for a toy project right now. In 3-5 years time I expect that it will be a very good choice, given that they've pretty much won and can hire all of the good engineering talent.
The parallels with MySQL are really astounding; I just hope that they don't end up being sold to Oracle or Microsoft.
The team at MongoDB really cares a lot about making the best database product possible. I knew it when I was there and still think so after I've left.
When choosing a database, caring is only one aspect of it. It's necessary, but not sufficient.
None of the criticisms I have ever seen or heard about MongoDB were related to Mongo Inc and their office culture but rather their product.
>The team at MongoDB really cares a lot about making the best database product possible.
... some what ironic.
[0]: https://docs.mongodb.com/manual/tutorial/resync-replica-set-...
> A replica set member becomes “stale” when its replication process falls so far behind that the primary overwrites oplog entries the member has not yet replicated. The member cannot catch up and becomes “stale.” When this occurs, you must completely resynchronize the member by removing its data and performing an initial sync.
> MongoDB provides two options for performing an initial sync:
> Restart the mongod with an empty data directory and let MongoDB’s normal initial syncing feature restore the data
> Restart the machine with a copy of a recent data directory from another member in the replica set.
Note: the second option is not a real option when you're dealing with a 700GB database. By the time you finish the copy the oplog will be too big anyway. Thus, making all these steps completely pointless.
And that's why it's so bad. They even acknowledge the "correct" solution is to rm your data and resync.
Its open to anyone by default, eats all memory and then crashes! I had it on and off my machine in less than one week and won't plan to go back.
n.b. I can't find the original "Call Me Maybe" post, but this later one [1] is similar.
HN Discussion: https://news.ycombinator.com/item?id=9417773
Any more information on this? How did he "solve" the problem?
Edit: straight from the horse's mouth [0]
It seems the problems were indeed "solved" and not solved.
In 2013, I performed an unpaid analysis, in my nights and weekends, of MongoDB. I found a bug leading to the loss of acknowledged writes with majority write concern. Mongo fixed this bug within a few weeks.
In 2015, I performed a followup test as a part of my work at Stripe. I confirmed dirty reads (which were already documented, though perhaps not well-appreciated), and discovered stale reads (which ran counter to MongoDB's documentation). MongoDB wasn't enthusiastic about that report initially, but got things sorted out and started work on adding majority and linearizable read concerns. I found that writes appeared linearizable.
In spring and summer of 2016, MongoDB paid me to help expand the Jepsen tests and hook them up to their internal CI system, so they could use the Jepsen tests to help verify their ongoing work towards linearizable reads. I privately confirmed that MongoDB still failed to prevent stale reads, but the dirty-read failures I'd seen in 2015 appeared to be prevented by majority reads.
In fall 2016, MongoDB announced they were almost ready to release 3.4.0 with support for linearizable reads, and paid me to perform a full analysis so they could be more confident in the results. It passed the linearizability tests I had initially written in 2015, but I offered to expand the tests to be more aggressive. Our collaboration resulted in the present analysis, uncovering design flaws in v0 and implementation bugs in v1. MongoDB worked to develop patches prior to 3.4.0's release, and that's why it passes now. :)
This is exactly the info I was looking for. Thank you.
By comparison, it's the difference between a bug report that says "you lost my data, you suck!" and "I've identified [x] ways your product can lose data and here's exactly how to reproduce each one."
I like it.
> "Once I've prepared a written analysis, the client may defer publication by up to three months. This gives the client a chance to understand the faults I've identified, attempt to fix bugs, update documentation, and deploy fixes to users."
So, a client can get the analysis done and attempt to defer publication for three months, after which time it'll be published. No?
There's no "standard practice" that I know of--very few people are doing this kind of work. Behind every one of these analyses is weeks of contract negotiation where I try to convince assorted lawyers & CFOs to go along with my weird, idealistic ethical policies. And conversely, those lawyers and CFOs do their best to balance the desire for correctness with their duty to protect the company. This policy outlines how we compromise.
Originally I did offer a client veto, because clients weren't willing to sign without some assurance of control over the outcome. My current standard contract for analyses actually drops that client veto: I have final say on the analysis content and publication. There's still a grace period to allow the vendor to fix bugs & get things in order. I'm adopting this as the standard going forward, but it's been a long road to get there.
That said, I do perform private consulting--usually for in-house systems, sometimes for clients that can't afford a full analysis, and sometimes as a precursor to more involved work. That means that yes, vendors may be aware of bugs and I won't have told you about them--but I promise that my public work remains honest and forthcoming.
Curious, is there any single book you would say you've found to be the equivalent of say CLRS but for the fundamentals of distributed computing paradigms?
My current plan for deep diving into distributed systems theory has just been to go through resources like "Distributed systems for fun and profit", and aggregated lists/overviews like https://henryr.github.io/distributed-systems-readings/ and your distsys-class notes and then just DFS into certain topics from there - compared to the approach I've been taking with deep diving into OS and database theory which is just to go through text books like Operating System Concepts and Database System Concepts while applying the theory in side projects.
I think that this really shows how mature of a Database MongoDB is.
Wait, it is me not understanding what the abstract is trying to say?
Giving aphyr some money to do this makes me think much more of Mongo now; I wouldn't previously have considered it for serious use but this is an excellent demonstration of intent.
Or that it took this long for them to pass basic proficiency tests. How do other database systems fare with these tests?
That being said, none of the big players in sql are there, so you can't size it up against postgres or mysql.
For example, Kyle integrated some serializability tests into the VoltDB Jepsen testing that wouldn’t apply to Mongo.
I'd say the marker of maturity here is that MongoDB has put significant time and effort into correctness: they take clock skew and network partitions as serious failure modes, they've redesigned their replication protocol, added options for stronger reads, and invested in their own correctness test suite, and Jepsen tests, as a part of their CI process.
I want to see more than just one test from more than one source and seeing their sketchy performance in the past only time will tell if this new version will be any better.
> I think that this really shows how mature of a Database MongoDB is.
Yeah no. It's a test. We have to wait and see in productions and see the data to prove that. It's also the first version to past this test. It means something but not a whole lot without real production data and other people attesting to it.
https://rethinkdb.com/blog/rethinkdb-joins-linux-foundation/
taken from their page : https://www.mongodb.com/mongodb-scale
This should give people some insight on why MongoDB is being used in the industry.
How is the performance with the new v1 protocol? (still faster, I'm sure)
Which would imply you have to restore from backups continuesly. Or else any bug you run into might be the result of data corruption.
Its the fact that you cant even tell. And its not just about loosing data. If you rely upon a consistent world view (i.e. schedule payment, jobs scheduling, vms provision or shipping, account systems) the risk is unacceptable.
I can't imagine developing any software that involves relationships between entities that does not have data consistency. Check constraints, foreign keys, and data type validation all provide a minimum sanity level of the underlying data that allows your mind to focus on more important things. Otherwise you're entire application is going to be littered with those same sanity checks.
I'm not saying that all apps need that type of data store or that there isn't room in this world for NoSQL stores, I mean specifically that complicated interdependencies and validation checks lend themselves well to the relational model.
The client will never experience Committed unless it really is committed, and thus never report that it was successful to the user.
At the very least I'd want to know exactly how much data loss we are talking about, which is probably harder to answer than just using a platform that doesn't suffer data loss in the first place.
Obviously this is true of many products, but how serious is it in practice? The criteria for 'good enough' can't be passing Jespen because very few db's do.
Storing schemaless json is a very valid use case. And the fact is there aren't really many proper NoSql alternatives, other than RethinkDB, which I hope becomes popular, and maybe Couch. e.g. Cassandra is often touted but its a k-v store, not a document db.
There's a reason Mongo became and continues to be popular. I still think Rethink is superior in every way.
What currently still makes MongoDB nicer than e.g. Postgres JSON are the libraries embracing schemaless while JSON in PG libs still feels tacked on.
Recently I had some bad experience with MongoDB support though.
No experience with RethinkDB.
I guess there are those who will say that even using batch-commits, that MongoDB could never have achieved the performance it currently does (by bypassing any ACID) if it was built on top of MySQL. But regardless, why not focus efforts on improving MySQL batch processing performance rather than throwing it all out, starting from scratch, and writing directly to disk. I know MongoDB became a success, but I think that is 'in spite of' their decision to start from scratch and not 'because of' it. Also think of the power that would be available if there were some relational table capability (true real MySQL ACID) right inside MongoDB whenever it was needed, if they were the 'same animal', rather than having to use two totally and completely separate DBs if you need NoSQL and also ACID in an app, which 99% of apps DO NEED, at some point, once an app grows beyond the round-one funding startup-toy phase and MongoDB falls flat in it's RDB capabilities.
Taking really old code and trying to make it do new things is very very hard. After you've invested all that time and effort making mysql better, it's still a database owned by oracle.
Think how Java would have turned out if Sun made it to be cross compiled to C rather than run in a VM.
There are plenty of examples in the Language world also: Go, Rust, etc. All those new languages SHOULD have built on top of Java, for the same reason Mongo should have built on top of MySQL. BTW, Java DID leverage C++. The Java compiler IS written in C. Mongo didn't use any part of MySQL, but Java uses EVERY part of C.
There are two challenges here. One is that MongoDB has a different data model than MySQL: hierarchical, schema-less documents instead of uniformly-typed flat tables. It's possible to map one to the other--look at Postgres' support for JSON datatypes. That involves extensions to both the storage format and query language, but it's certainly doable.
The bigger problem is that MySQL's local transactional isolation is nice, but not particularly helpful in a distributed context. Distributed transactions--even single-document ones--are still tough to implement correctly and efficiently. As an example, consider Percona XtraDB, or MySQL+Galera Cluster. Both are building on a serializable single-node system--but in a distributed context, they wind up failing to provide snapshot isolation, let alone serializability.
Just use Postgres (or several other options) then? It has all that built right in if it's what you need. But...
> if you need NoSQL and also ACID in an app, which 99% of apps DO NEED, at some point,
...I doubt 99% of apps really need this.
> once an app grows beyond the round-one funding startup-toy phase and MongoDB falls flat in it's RDB capabilities.
Or don't try to force MongoDB to behave like a RDB and you won't hit those problems? I just moved a system from Postgres to MongoDB, and it's running faster on way cheaper hardware now. Not because either database is inherently better than the other, but the use case lined up with mongodb perfectly and the old model was leveraging Postgres really poorly. Eventual consistency is fine, and I can denormalize certain data for faster reads because I know it won't be modified later.
I think relational-type queries are so common that any significant app will need them. For my own side project (meta64.com) I'm using JCR-SQL2, on Apache Oak jackrabbit, so I get to do lookups fairly easily, but it sure would be nice to have a full-blown ACID RDBMS engine sitting right there to use also, in the same engine.