MongoDB 2.0 Should Have Been 1.0
luigimontanez.com
luigimontanez.com
I try not to take the actual comments or articles as law, but rather use them as a temperature reading to figure out where in the slope we are currently for a given tech or trend.
Given that Mongo-talk has absolutely dominated HN over the weekend as well as reddit/r/programming, I am interpreting this as being in the bottom of the dissolution curve right now.
It is this point where the community push-back and temporary "hating on" forces the team to go into overdrive, addressing whatever pain points the community has griped on the loudest, in this case:
- Write locks
- Durability / Replication consistency (already addressed)
I imagine Mongo 3.0 will represent the final climb out of the dissolution curve where all open complaints have been addressed and we actually get back to solving problems with the technology.
I don't know that Cassandra or Redis or CouchDB have completed their hype-cycles yet, because they haven't had the hyper-aggressive response from the community during the dissolution step. They are all popular and well-liked, but it seems like their popularity is still climbing.
It is all interesting none the less. Mongo is a huge success, regardless of how many of these articles are written.
I've not seen a team as dedicated and involved like 10gen is for a long time; Eliot still answered 100s of messages on the group every week (Along with every new member of the team) -- which I find the biggest indicator of Mongo's future success. If the CTO is carving out that much time during the day to stay involved, while still bug fixing, replying to posts like these and testing bug reports... that's a lot of love right there.
As far as I can tell, no one has hated on Redis or HBase (except for the brief period when antirez tried to add VM to Redis) because they both (a) work and (b) solve real use cases. Has there been any suggestion that Redis or HBase lose data?
However, maybe you are right in a more general sense. Do you think that the idea of NOSQL itself is reaching the trough of disillusionment? Are we seeing a shake out of which of these data stores are actually designed by people who know what they are doing, both in (database) theory and in (systems coding) practice?
I would say Mongo is the most popular NoSQL data store at the moment; whether it is mindshare or deployments and that is what caused the move along the cycle so much faster.
I don't mean to detract from any of the other NoSQL projects; they don't have the marketing or manpower budget that 10gen has so I wouldn't expect them to be at the same place in the cycle. MongoDB came on the scene with the only NoSQL solution that promised SQL-esque queries, insane magnitudes jump in performance AND a big commercial company behind to. To anyone trying to understand "NoSQL", it was the clearest and safest place to look.
Since then we've seen the cracks in that original argument (fast and unstable means terror in production), and 10gen has changed focus as needed and addressed those. During that time it wasn't just coding like a lot of these other projects, they were putting on conference after conference, garnering mindshare and getting developers on board.
An open source Apache project just won't move along a hype path as quickly as a force like that.
(I am making no statement towards quality, performance or worthiness... just positions on the hype-cycle).
> Do you think that the idea of NOSQL itself is reaching
> the trough of disillusionment? Are we seeing a shake out
> of which of these data stores are actually designed by
> people who know what they are doing, both in (database)
> theory and in (systems coding) practice?
The early days it was so exciting to see different ways to store/retrieve data. We had been with SQL for decade(s) and it was very exciting to see something new/fresh and fast popup.
Then everyone started storing data every which way they could think of.
Then a few of us starting solving problems with those new ideas... so far so good.
Then some of those projects and new projects built on those new techniques blew up in popularity, and suddenly the "real world" came knocking and we started to actually test the metal of these things in production... with disk failures, network failures, power failures and administration failures.
Like shaking out a rug, the weakest approaches got shaken out and the strongest teams/products weathered the storm to grow stronger and more stable.
2011 was the year NoSQL "Grew up", I imagine 2012 and 2013 will be the year that NoSQL comes all the way out of the dissolution curve completely and, in a metaphysical sense, "goes into production".
I mean that in the most hand-wavy way, not literally... literally LOTS of people have it in production.
I mean it in the sense that you stop seeing articles like these that sparked all the Mongo hype recently or articles about horrible shortcomings or failures about XYZ datastore.
Early on the teams making the NoSQL solutions AND the users didn't really understand where this boat was going or how the puzzle pieces fit together... they just kept working and refining.
This entire year we've seen more and more specialization in the NoSQL community:
- Antirez gave up on data-larger-than-ram approaches and
wants to focus Redis on what it is amazing at: being
fast, in memory.
- CouchDB, building on its uniquely awesome m-m
replication, moves into the mobile space with data sync
solutions that are awesome.
- MongoDB keeps replacing MySQL in production at many
large-scale startups in the valley; showing more and
more the exact migration path to take.
- Cassandra becomes markedly easier to use with CQL and
combined with its CouchDB-esque replication behavior,
suddenly makes all sorts of sense in densely populated
deployments.
This seems silly in hindsight, but I don't think we or the teams really honestly knew where this trip was taking the technology a year or more ago.
2012 will be a year of polish, stability and deployments.
2013 will be production deployments and replacing MySQL in more and more places.
2015, it all starts all over again as SSD-optimized data structures and data stores revamp our understanding of databases :) -- I am half-kidding.
That's my 2 cents anyway.
Cassandra and Riak have a similar replication model -- the are deployed into a "ring" and the data in the ring distributed across some (or all) of the nodes depending on your ReplicationFactor (how many nodes to copy each piece of data to).
If you query for a piece of data that a node doesn't have, it hashes the query and routes you to the node that does have it.
CouchDB is a bit different, in that by default it treats every node as a master and replicates it in its entirety to any other nodes registered as a replication target.
You can shard with something like BigCouch, but that is 3rd party.
This is different than Mongo which is master-slave-slave-* or Redis which I believe is master-slave as well (I never got a clear answer on how "slave" nodes in Redis resolve or push changes back upstream to the master).
In my experience there is no way that this is possibly true.
This is not directed at my parent comment but, seriously, this is getting tiring. Comparing NoSQL to SQL is like comparing a rubber mallet to a hammer. Sure, both might be good for some of the same things, but each has its specific use case.
I think it's pretty well established that MySQL was the default data storage system (I say to include a broad range of systems) for web applications in the open-source world for a good chunk of the last decade.
And there's at least some reason for a default to exist. There are many applications where the author(s) don't have particular data storage/management expertise, and they'll be looking to use the "best practice" or "default" system that everyone else is using. So it sounds entirely reasonably to me that there will, again, exist a default way to store and manage data.
And it also seems natural that various systems will vie for that title, because there are a huge number of potential users there. Others will avoid that title because they want only experienced users to be involved (which I think is misguided, but it seems there are always a few).
So, I agree with you in the strict sense that there's no bestforeverythingontheweb datastore, and it's way too early to assign that title to anyone right now, but striving for broad appeal is certainly a reasonable thing to do.
I have seen first hand some of the issues raised, we've had data disappear, recurring random crashes, ect. But I think the difference is 'everyone' knew that there were issues with MongoDB, you just needed to check in jira. Jumping to 1.0 too early is clearly part of the reason for this backlash as not everyone thinks to check the issues because they've come to believe 1.0 means its ready for mass adoption.
That being said, I love MongoDB and I would still use it in other apps, just got to decide if it's the right tool for the job.
https://github.com/apache/couchdb/blob/trunk/etc/couchdb/def...
Once the prototype was up I started working on other parts of the system (and the business for that matter) and only half-paid-attention to the mailing list.
The mailing list for couch is quite good, btw.
I think it just came down to preference, MongoDB has a nice way of doing quieres, without having to use views, also its a bit faster, although I've seen people speed up couch by using protocol buffers etc. Now that I havent been following couch for a while the fragmentation in between couch versions, doesn't help when evaluating if I should try it again.
That's the fallout of eventual concurrency, though.
From now on, for all my open source code, I'm going to version with this translation table:
Alpha: Beta
Beta: increment version by random number between .1 and .2, eg: 2.0 becomes 2.1.7
Release Candidate: add the word "Enterprise Edition" to the version
Release: Add the same letter Oracle added for that version: 8 -> 8i, 9 -> 9i, 10 -> 10g. The exception is anything that happens to be a BMW model, in which case I'll just make it the same as the BMW. 3.2.8 -> 3.2.8xi, 4 -> z4, &c.
Oh, and there's a chance that yesterday's drama was a hoax: http://news.ycombinator.com/item?id=3205573
Edit:
Associating special meaning with these things has always been "gamed". Access went from 2 to 7. Heck, office went from 97 to 2000!
There are places to fuck around with your stack choices, but databases aren't one of them unless you absolutely need this new tech. Lets be honest and acknowledge that most sites using Mongo today could be using an sql based solution with no issue.
Some people are leveraging it for a reason, others.... just for kicks. If your doing it just for kicks, you'd better be comfortable with uncertainty.
Latest developments around MongoDB remind me the history of Oracle DB described in "The Difference Between God and Larry Ellison..." book: they basically were selling a DB that was far away from prime time. I'm not surprised at all that people may run into the issues with bleeding edge software - I figure that's the price that early adopters must pay anyway.
But Riak is a Key-Value store, not a document one. If that's the premise I wonder how illuminating the rest can be (I kept reading: it's not)
But they fall squarely in different categories: Riak KV, MongoDB document (and HBase Column, Neo4J Graph for example)
"For better or worse, many people consider MongoDB and Riak to be competitors. In reality, there are very few similarities between the products."
http://seancribbs.com/tech/2011/11/07/mongodb-and-riak-in-co...
[citation needed]
Suggesting that the 1.0 version number should have been reserved until more recently makes me think that what the author of this post is really saying is something like this:
"MongoDB wasn't really production-ready until recently. People who wanted to test something bleeding-edge out in the real world should have still been free to do so, but branding the product as a beta and giving it a sub-1.0 version number would have helped set expectations correctly."
As a Java programmer, I've not really heard a particular level of criticism of the OpenJDK project. Java the language sure does have its fault.
Although I suspect you were just Java bashing.
[ed] s/it's/its/g
Wow, that's a bold statement. I don't think I'd go out on a limb like that considering that 2.0 has only been out for a couple months and doesn't have a lot of production users.
http://www.google.com/insights/search/#cat=0-5&q=cassand...
I added Cassandra, as it's substantially more popular than Riak (whereas HBase is not, and you only get 5 slots).
Some say not yet: http://news.ycombinator.com/item?id=3202028
I used MongoDB last year and it worked fine for me. (I maintained it for about 9 months.) But of course I can't generalize that to other people's experiences, so YMMV. ;) I just used it squarely in the use-case which everyone mentions — many "well-behaved" writes which occur when no one's reading. Multiple replicas. Leveraged its indices. It wasn't an authoritative source of data, so in principle I could repopulate it. Dealt with failure modes, so missing data wasn't catastrophic.
I think articles like this are useful; when evaluating software, one thing I do is assume it's buggy and its proponents are deceivers. (Whether or not they intend to deceive. I can imagine being corrupted by wanting something to be true.) So among other things, I hunt down criticisms. Had this article been around last year, I'm sure I would've found it a useful hub in doing this due diligence.
(The actual thesis, of what the version number should be, is not so important to me... Version numbers are in some sense arbitrary.)
Original post was this by nomoremongo:
http://news.ycombinator.com/item?id=3201772
Post actually discussed was:
http://news.ycombinator.com/item?id=3202081
Very clever.
The way I understand, apparently nomoremongo wrote it but it was reposted quickly by nmongo (http://news.ycombinator.com/threads?id=nmongo) in hopes that they could then become the top post, so later on they can yell in all caps how it was a hoax and thus discredit the original post.
You know nmongo, if you are trying to help MongoDB you just did the opposite. I've said this many times here before, but the best marketing is your competitors stupid marketing. You are providing stupid marketing for MongoDB and you are hurting it.
> People should note that yesterday's anonymous pastebin "Don't use MongoDB" article was apparently a hoax,
Now another question isb do you know that it wasn't a hoax but continue in the same vein in hopes to still save the day, or you actually believe it was a hoax?
I know exactly what it will take for me to believe the pastbin story.
I'm curious, what will it take for you to not believe it?
At the same time, it got on the front page because the story resonated and made sense to others.
There were quite a few people who commented how "oh yeah I've had problems with lost data". And I think that is what pushed the post's popularity more than the original pastebin. So the discussion got a life of its own after a while. Followed by response posts and response posts to those and so on.