Why Did So Many Startups Choose MongoDB?
nemil.com
nemil.com
10gen is not a company dedicated to databases, at least not if the initial code quality of MongoDB is a signal (and it should be, since if you know a thing or two about storage you wouldn't take the compromises it initially did)
Unfortunately, many fresh graduates with no experience in storage decided to evaluate the technology based on its very visible merits, without considering that scalability is not really a top priority even in companies that have to "scale", and that reliability trumps every other metric.
Mongo has failed spectacularly in being a reliable data store, and its scalability is mediocre, given that it's a lot more about query planning, analytics and optimization than having out-of-the-box sharding.
This is one of those areas having some seniority does wonders. Storage concepts are not easy to grasp, and evaluating a database's reliability takes much longer than most product cycles.
And you might want to look at how SalesForce has implemented a highly scalable architecture on top of Oracle RDBMS servers. That includes a document store as well as a dynamic denormalized schema layer that works real well for reporting.
Heck, I've implemented a persistent memcache server on top of SQLITE and it scaled pretty well because modern servers and network infrastructure just happen to do most of the work needed.
It is not easy to learn PostgreSQL well enough to evaluate it, but more and more people are doing so and finding that it is the ideal open source persistence solution for the vast majority of business uses.
I'm not insulting Postgres (actually a big fan), but Postgres added JSON support after. By that metric, document store in Postgres is less tried-and-true than MongoDB. Does that mean it's the wrong choice? No. Age of product is not always a good qualifier.
Whatever else you might say about Postgres, that community cares about the integrity of its users data, over pretty much every other consideration. I don't think the same can be said of Mongo, or any other "NoSQL" anything.
I feel pretty good about DynamoDB, but the development model WRT to indexes and throughput is what keeps me away.
Except it's not and age of the product is a bad qualifier.
The MongoDB hate hype train is even worse than the MongoDB hype train ever was.
Unstructured documents are very rare. Querying arbitrarily against them even more so. For the fields that do need querying in a document, put them apart and add a few indexes and you're good. That's been my experience and I've been around a while.
And if you do not want to commit to either SQL or NoSQL, the PostgreSQL has you covered. You can put a JSONB column in any of your tables to handle the part of your data that is schemaless, and you can still index the data in that JSONB column.
There are a very few applications that need distributed scalability beyond what Oracle or PostgreSQL can provide, and those ones need NoSQL. They also need highly skilled competent developers who are capable of reimplementing some needed functionality of RDBMSes at scale.
There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
You also do have to think at least a little about your JSON structure given you can't really use an ORM to query JSONB, and the syntax for JSONB is a bit awkward at times.
It's much much harder to design a reliable engine than a new datatype.
> There are few applications that need distributed scalability beyond what PostgreSQL can provide, but I'd argue that it's not uncommon to have problems with Oracle scalability, if you don't have deep pockets for licensing.
Yes they are, but chances are you don't have that need. You're not Google or Facebook.
You also picked up a really bad database to talk about scalability[1]
[1] https://www.datastax.com/wp-content/themes/datastax-2014-08/...
SQL is a lot for newcomers to grasp on top of learning the rest of their stack. The appeal of "no schema" is most likely what gets them as it seems easier than SQL. Simple things at first seem daunting like: constraints, joins, and indexes. What they fail to see is that doing these database level things in the application layer becomes the final level hell and you get to live it.
People also undervalue their data and it's integrity.
I personally do not like mongo and see zero reason for it except perhaps in some very niche use cases and even then, I'd still choose another document store over it.
I'd say don't worry: I almost never use anything I learned in school.
The Java was to shallow as was lots of other stuff.
School is just there to convince your first few employers so you can het a chance to see how it is actually done.
Oh, and to give you the credentials to stop annoying sales guys that are about to convince your boss that "you need Active Directory because your network is slow". (Guys company had botched a certain server install and tried to solve it by selling even mire stuff. )
Red brick Russell Group British university, not prestigious.
The classes were not mandatory so you could technically graduate with CS without knowing anything about databases.
I did. I wouldn't use it again. But reliability just wasn't an issue. There was a time when there were insane defaults that made this an issue, but that's a long time ago.
Channel: chat.memsql.com My UN: eklhad
To put it another way, RethinkDB did extremely well on Hacker News. Twitter didn't, if you remember all the negative posts (and still went public). There is little relation between success on Hacker News and company success.
To be clear, I've only ever run it as support for something else that depended on it. It just isn't well-suited to my projects and it wasn't interesting enough to invest much more than noodling a bit with it. I'm sure it is a fine product that turned in to a victim of its own hype.
It does serve as a decent example for the next generation of developers about how flavor-of-the-month trends in tech make people do stupid things, like decide they don't actually need ACID for handling money.
It's in vogue to hate on PHP especially here on HN. Language hate based on some sort of popularity contest is silly and language snobbery wont be tolerated at any company worth its salt (read: NOT most startups)
If you think PHP is looked down on by those who care about language features because of a popularity context, rather than because of asinine language decisions and limited support for modern programming paradigms, I think it's you who is missing it. Since any program can technically be written in any (Turing Complete) language, the fact that some big sites were made to work with PHP also sort of misses the point. When most of the big "success stories" adopted PHP, options for web-friendly open source languages were much thinner. Now if you look at the new development projects they highlight, you don't see PHP come up that much. Facebook, for example, is prominently using Haskell. I don't know anyone who both programs in Haskell and is willing to touch PHP.
Of course, many thoughtful criticisms of PHP have been discussed on HN. e.g. https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/ & http://poincare101.blogspot.com/2011/11/php-sucks-but-some-f....
Still, that's not going to do much to change public perception, and I guess that's somewhat fair. PHP has earned it's reputation, even if many (but not all) criticism are now largely dated.
I'm just happy I switched to working on other languages, aside from the occasional Laravel-based side projects.
Is public perception the reason you are happy to be mostly using something else now, though?
I really enjoy some of the more flexible syntax and design choices present in some other languages. It also help that various job requirements over the year have required that I build a broader skillset. Finally, the growth I've experienced by stepping outside my comfort zone the past few years is probably the best benefit, which is a good experience regardless of which language(s) you start with.
In the end, however, as we both know, most decisions in software development are an exercise in managing tradeoffs. I guess the core of my argument is that, in the past few years, PHP has done a great job at improving its balance of tradeoffs.
I was curious and wanted to see if that is the case. So I checked for the most basic thing. The arrow notation for lambdas. Surely they must've added that by now. Surely they must've added that before Java did. Nope, still RFC. Now I wonder if it's idiomatic to pass those "long form" lambdas around when writing in functional style or is it something nobody does. In any event this little discovery makes me think parent's sentiment seems to be still justified.
Personally, having had to write a couple of Mediawiki plugins and an extension for a rubbish CMS called TYPO3, my experience so far is that the PHP hate is very much justified.
Seriously, the biggest problem with php is that fundamentally it's a templating language. One thing I've seen with templating libraries in general, is that they are meant to take a template and a data structure and display output based on these 2, but what ends up happening is they end up converging on turing completeness. It starts with the need for some kind of looping structure in the template. And then someone needs some random logic that is easier to do in the template than before the model, and so another feature gets added to the templating engine. And so it goes, things get kind of bolted on, without a grand vision, and you end up with an API that is inconsistent, where the order of inputs is never the same (is the first parameter haystack or needle??).
Also, the basic shape of the code that tends to get written in PHP is heavily influenced by it's being a templating language. PHP applications tend to end up having a lot of code in the template, that should be done before rendering the model. Rather than a sane 2 pass system, where all the data is gathered and the logic handled to produce a simple data structure that is passed to a very simple template, you end up with crazy monstrosities where there's opening and closing <?php> tags everywhere. And you render part of the page, then grab some more data, then render some more page, then maybe do a curl to a 3rd party vendor, then render some more of the page, and it's actually very hard to keep switching mental state to match the context of the code you're looking at.
Anyhoooo, you're getting downvoted because there's lots of literature on why PHP is a terrible language. I'm only replying because it's sort of cathartic, since I maintain a wordpress codebase, and any time I have to interact with core, instead of being able to sanely intercept the underlying data, modify it then pass it along in one place, I have to constantly add lots of random hooks in lots of random places to catch the data just before it's being rendered. It's a total mess, and far more complicated than it needs to be (though there is some debate whether this style made it more approachable from a novice plugin development perspective).
By that time I was doing most of my coding in Ruby and Python, but this businesslike approach to PHP made me realize this is a place where people really get their hands dirty and build things, rather than care about fads.
Things have really changed, the tone & level of discussion here are now very different. Seems like PHP is kind of a canary in the coal mine in that regard.
It's more like those vast parts of the internet are working despite PHP. Why do you think Windows is powering vast numbers of desktops? It's not for its technical excellency. ;)
So, it started as a shitshow on the back-end and people who had to maintain systems based on it hated it, but it got popular because it was easy to get simple systems up and running with. Then, over time, it slowly became less awful as it was substantially rewritten and large portions of it redesigned. I'd say PHP is a very, very good comparison.
Reminds me of: https://slack.engineering/taking-php-seriously-cf7a60065329
You make that sound like its not a big deal vs vanilla php.
The choice was narrower then. Ruby and Python, while already active, did not have nice web and DB frameworks (both RoR and Django started in 2005). Java was also much less mature, and even more wordy.
PHP, on the other had, was easy to get from any shared hoster.
OSM started in 2004, but the editing API was ported to Rails in 2007ish.
Downloads can be repeats, no? How to discern real number of productive installs?
Skills aren't objective, are they?
Revenue, great...but they won't disclose it.
For a long time all I had ever done was relational, but then my company started using Azure table storage for a lot of projects and I became familiar with other models. Then I discovered Mongo while catching up on Angular, and enjoyed using it with the mostly small apps and prototypes I build. I don't know it well enough to try and support it's use on a large scale. After seeing how poorly designed the data is for so many companies out there, it's not a surprise to hear poor choices were made (and probably some poor execution).
Current Indeed.com Job Trends data actually show better numbers than the 2102 stats, but nonetheless show MongoDB jobs in decline.
When 10gen came along, they sold us an amazing vision. We were processing 500 million to billion social media messages a day and they sold us on this dream that we would have very fast writes, a cluster of 9 machines in master-slave-slave config for fail over. We wouldn't have to write SQL anymore (which was actually funny to watch one of our engineers try to unlearn table denormalization).
At the end of the day, it was hype. I had to stay up during multiple 2012 debates for 5 hours and flip the Mongo databases because they kept crashing. The shard key they set up for us was slightly out of balance so the data distribution was 30-30-40 and every so often, that server processing 40% of our data would jump to 50% and then knock that server offline leaving the other 2 to split 50% of the work... knocking them offline. There were also tons of replication problems all traced to write locking. At the end of the day 10gen solved one problem, but our company traded that solution for other problems.
That experience taught me that you really need to understand what you're trying to solve before picking a database. Mongo is great for some things and terrible for others. Knowing what I know now, I would have probably chosen Kafka.
Going from sql to nosql would be denormalizing data not normalizing it so, the engineer would be unlearning table normalization.
In order to leverage the multiple CPUs in a box you'd need to run multiple instances of mongo on the same machine and we designed a whole system around sandboxing instances.
But then you'd run into memory allocation issues and how badly it would handle indexing and handle running out of memory, which mostly involved crashing.
We ended up having to run about 100 bare-metal servers to support what would probably have been a level of queries about 10 PostgreSQL boxes could handle easily.
Truly a terrible DB.
If it is all in mongo? Welp. How is your python? "See there is this key, and you need to sum this -- OH and remember that this does.. Hmm, "
Part 2 will dive into the benefits of Mongo along with the mistakes that some teams made with Mongo. Part 3 will dig into Mongo's marketing strategy and how it influenced uptake. Sign up here to be notified: http://eepurl.com/cxs5Zr
As any reflective engineer, it's important for me to provide a thoughtful perspective backed by evidence. This can be a fraught discussion and I personally know teams that truly value Mongo, and other teams that had tremendous issues. I want to provide color about the early years of Mongo (not today). My key point is that as engineers, we have to get beyond hype, thinking critically and choosing the right tool for the job.
I have a background in enterprise s/w development; we always lock down on most of our requirements before we start writing code. RDBMS is fantastic in this scenarios. In my startup, we build something light and show it to bunch of prospects and iterate. MongoDB fits this mode of s/w development for us. Sure there are gotchas and other issues but they have not affected us yet.
- There is a new replication protocol for replica sets, improving the election process.
- WiredTiger has been introduced and became the default storage engine.
- Journaling guarantees durability.
- Sharding architecture has changed. Config Servers are now deployed as Replica Sets and some of the tasks that used to be managed by mongos now are managed by the config servers.
- Storage efficiency has been added through compression.
- Security has been improved with encryption at rest.
- The aggregation framework has more options for BI.
- There are a variety of different indexes that can be created to improve query performance.
- and finally, stop saying MongoDB is schemaless because it is not. There is a schema in place and this schema is flexible enough to be polyform.
These are features available today in 3.4. You might have issues in the past, but if you really understand the newest architecture, you are capable of seeing how much MongoDB has grown since 2.4.
Compared to mysql I like that it has automatic failover out of the box. I can also just add new replica set members without having to stop the master or bring a slave offline to rsync it over.
With something like mysql I have to learn about & choose between row based replication or statement based replication. I have to choose between binary logging or global transaction ID logging. With Mongo I just call rs.init() and rs.add() and it "just works".
Perhaps I'm just not hitting the level of traffic where the problems begin to surface, or maybe postgres would be better... but its been great so far. No complaints. The only issue was de-normalized data getting out of sync due to bad coding in our app, which i do not blame Mongo for (and was simple to fix).
An app full of queries like "Get me all the content of slide Y of deck XXXX... or don't if the current user YYY doesn't have access to that" would make me itching for indices and foreign constraints across what definitely feel like relations to me.
And for the non-relational content blobs, Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images", "who are the top 10 users in terms of edits to content tree items, and what subscription level do they have" pretty neat and fast.
If you're in early stages such that you don't want to hire or acquire DBA skills, you're probably well served by offloading replication/backup/verification (which are all very possible with RDBMS) to e.g. AWS's RDS product.
HA is important, is covered by replication+failover. Periodic backups are important, are covered by a export+rotate schedule. Point-in-time restores are important, are covered by storing replication logs. All are covered by a managed RDBMS, so you need to know what they are and how they work, but not implement or manage them. Most of the NoSQL deployments I've seen just rely on the out-of-the-box replication, and get HA but that's it. The engineers I've seen that don't want to understand the simple high-level ramifications-to-developers of RDBMS replication don't want to understand ramifications-to-developers about their NoSQL replication either, and they take a risk of encountering those ramifications needlessly unforeseen.
As the data grows, yeah, maybe you'll eventually hit technical or economical limits that mean you'll want to manage your own RDBMS with an application-tuned replication and sharding scheme, so you'll want to hire or train at that point -- but there's huge huge room to play within the economic and technical characteristics of modern managed cloud RDBMS-as-a-service for any app that has a hint of a business model, given that we measure the cost of such services in cents per billion unit-hours.
But that's not a query I ever need. I load the user's document, which contains an array of slide (embedded documents). There is no relation between documents & slides, slides are part of the document.
But take it one step further & imagine you have a variety of widgets that can be placed on slides, with no common schema between them. Your choice is model out 100s of tables, use EAV, or go noSQL. We did the latter.
> would make me itching for indices and foreign constraints
Mongo has indexes. Regarding foreign keys it does not have that, but usually you can just embed the document in all the places you need it (de-normalization). Which can be a problem, so if you don't want de-normalization you should definitely use a SQL DB, not mongo.
> Postgres's partial indices + indexes on expressions would make some fairly intricate reports like "since we launched the image embedding feature, per week, what percent of created or modified slides contain images"
Mongo has aggregation which has fortunately been able to perform all the reporting I've needed so far. I'm sure postgres is nice as well.
MongoDB feels like a different product to what it was when the hype was high. The company, leadership and engineering quality certainly have changed.
Someone else commented about the HN bias, which we can't deny exists.
I've been running an 1-member replica for as long as I could figure out how replicas work. So I've had my Mongo database running for the last 5 years. It's met the sweet spot that I was looking for when I knew less than what I do today.
Even if one were to rubbish the product, it still has value as the company seems to be doing well (even with the negative sentiment and press). Mongo sometimes feels like how "corporates" (at least in the recent past) feel about open-source. Dealing with our SAS account at work, the sentiment used to be that open-source was a waste of our time. Open-source is brittle and slow, rather use Oracle, etc. SAP HANA will change your life, in-memory is hard, our "appliances" run on unsecular hardware ...
Those sentiments have changed over time, so we shouldn't eternally write things off if they're healthily maintained.
I look forward to mixing SQL and NoSQL well into my future. DocumentDB wouldn't be what it is today if there was no Mongo . The SQL vs NoSQL flame wars get us nowhere to be frank.
It was a perfect mixture of this unfounded fear with an advertised simple solution. Add the idea that there is no need for a DBA, encourage developers to see Mongo as a magic data storage and nothing more, and you have the perfect product for those who don't feel the need to look deeper.
Startups seem more optimized to fast iteration and quick development. It seems to me that they chose Mongo because they prioritized iteration and ease-of-development over the hard-nosed style needed to use a relational database well.
Maybe it's an age thing (or the product of "bootcamps"), but I don't think that's true. I'm 40, and when I was learning, you connected to and wrote SQL against the database. These days I still spend a significant amount of time writing SQL, both for ad hoc queries and for complex queries / performance optimization.
> Add the idea that there is no need for a DBA
With many cloud database offerings this is essentially true. As for things like indexes, etc... developers targeting Mongo need these as well.
> encourage developers to see Mongo as a magic data storage and nothing more
In modern frameworks, this tends to be true no matter what the data store is.
1. Architecturally is was very familiar to traditional relational systems like MySQL. Meaning classical master/slave, sharding and absolute consistency via locks.
2. Mongo's rise coincided with the rise of NodeJS and continued on the success of Ruby on Rails.
3. Mongo has done a phenomenally fantastic job attracting developers with exceptional documentation and getting started materials.
These all combined into a perfect storm attracting legions of developers. Note that these things have absolutely nothing to do with the technical capabilities of Mongo as an actual database.
In point #2 substitute PHP for Node and RoR.
In point #3 just change the name from Mongo to MySQL. MySQL AB was brilliant at getting MySQL into Linux distros. When you came to need a DBMS to go along with PHP chances were you could just type 'mysql' and you would connect to a live DBMS instance on the host.
The 10gen team also imitated the MySQL playbook explicitly with built-in replication and an early lack of focus on ACID properties.
I don't think there's anything that should immediately preclude the use of MongoDB in a production environment. I personally wouldn't use older versions of it as a single source of truth, but there are definitely some great use cases for it, such as allowing developers who only know JavaScript to write non-critical backend services.
People seem to forget that database consistency is not binary, but residing on a spectrum. Like any other trade-offs, you should carefully measure the costs and benefits for your particular situation before making a decision.
The lead dev had never actually used it, but liked the idea of not having to come up with a schema and he really liked JSON as he was very familiar with it.
By the time we were done, we were easily one of the top 5 users of MongoDB at scale and it was responsible for 9 out of 10 of our outages due to various bugs or optimization issues.
It's like complaining getting graphic cards working because of drivers is difficult on Ubuntu, based on experience with linux in 2005.
I wouldn't be surprised if half of the commenters haven't used Mongo, or stopped in the 2.0-2.2 days.
A lot of software fails, companies close, while some succeed. Some software that has been hyped up ended up not succeeding, but it doesn't matter. One learns from others' mistakes and keeps growing.
You would be correct, as the mountain of ignorance about MongoDB on display in this and all HN threads on MongoDB attest to.
Yes, migrations work for RDBMS too, but with Mongo you only write migrations for data changes, not schema changes - which means a lot less migrations.
The option of having documents with embedded objects/lists makes it a pretty natural mapping to Domain Driven Design concepts, like aggregate roots + aggregates. One might even argue that while following DDD, it's preferable to handle most rules at the application layer and use the database simply as a performant, low-maintenance storage service. I know some folks here might view this approach as a sin, but it works pretty well too, especially for smaller teams without a dedicated DBA.
Maybe it's just me that haven't ran large enough systems to find the kind of issues with mongo that people always rant about, idk. Honest question: considering all these advantages above (from a developer's point of view), are there good reasons for choosing something like PostgreSQL over Mongo for typical business apps?
My thoughts on this pattern are to still use a robust, reliable db (Postgres) behind a message queue for the event store. Postgres does almost all of the things I need with the reliability that I need while letting me sleep at night.
An event store absolutely must be the most reliable part of the infrastructure, IMO.
I agree a robust, reliable db is super important - but isn't Mongo too, at this point? Even if it wasn't back in 2012? I thought most of the reliability issues and bad defaults were fixed by now...
In my planning for problem domain changing, personally I'm going to be writing a (relational) schema for the new data model anyway, which will be Postgres, and playing my event store over it and doing one-way writes to it going forward anyway. I'd rather reduce the mental maintenance overhead of two different databases to one.
If you're building your relational db based on the event store, I'd imagine even something simpler like Mongo would do it, but I agree that having a single db to maintain is an important advantage.
TBH, I'd rather write SQL directly rather than use an ORM though. I never want to use an ORM ever again. At least that's one problem that MongoDB solved.
I will never understand this attitude.
Why did you think you can only use SQL with an ORM? And if you didn't think that, then what was your point?
When it comes to Mongo, I think it's nice to have the DB schema documented somewhere but you don't really need to have it enforced with Mongoose... It introduces more problems than solutions.
With Mongoose, you have to run migrations on your data more often which slows you down. Though it might make sense for bigger companies.
Mongo did replace Oracle product but it was Oracle's Coherence[1]. There wasn't a concern of Mongo loosing data, because the data could be reloaded again from Postgres if it got lost.
I also heard that they were considering replacing Mongo with Postgres as well, but I do not know where this went.
I think 10th Gen is just good at generating hype. The migration was not because Mongo was better but because Oracle was expensive and not that much better from Open Source alternatives.
However, most people think NoSQL as NotRelational - when there is no ACID guarantee. They are choosing availability over consistency.
I'm pretty sure that's wrong and it always meant not to use the relational model; it was only “NoSQL” because it's short, easily memorable, and there's a percieved (though not exactly true) equivalence between SQL-based and relational databases.
http://jimgray.azurewebsites.net/papers/thetransactionconcep...
You do realize it's 2017 now, right? Many things have changed in MongoDB (and in the IT world in general) during the last 5 years. MongoDB is a mature, established, full-featured database - recognized by Gartner as a "challenger" to more established RDBMS vendors like Oracle and Microsoft.
MongoDB has many built in features that are extremely attractive for building modern applications. Built in HA & DR, horizontal scalability, multiple storage engines, rich indexing and query capabilities, enhanced BI features in aggregation framework, and many more...
And at the time, most of the mindshare is/was with mongodb for document-oriented technology.
This reminds me of every single javascript coder I have worked with. Just because they can get something running within a day, makes them look down on other coders and gives them some sort of complex.
No wonder the whole javascript stack is a complete and utter mess!
Relational db's are hard. You have schemas and query planners and the truth is, you have to take care of every single query. Every query counts.
The same query can take 30 seconds, or 30 milliseconds to run, depending on it using an index or not. You think it's ok for it to take 2 or 3 seconds, until it hits production and halts the server.
Mongo doesn't have any of that. Just load and dump json back and forth, which is perfect for fast prototyping.
Who cares about data integrity or technical debt?
Mongo has query planners, the same thinking you do in SQL applies to Mongo, the edge cases are different. For example, you can't have composite indices on multiple arrays. The same principles we consider when tuning our RDB apply to Mongo, normalisation comes at different forms/levels. The same applies to denormalisation. Just because there are/were "no joins" didn't mean that everything goes into a single collection.
I run some Mongo deployments, and I take as much care in understanding performance, as I do with our SQL deployments. Databases are hard to those who are unwilling to open up the manual and read. It's not only SQL.
My favorite recent one was a company that had to persist their Mongo database into AzureSQL (nightly job) so they could then push it to Good Data for analytics. They could have very easily used SQL Server and pushed directly from there to Good Data. And yes the company admitted that was the case.
Also the table is misleading, PostgreSQL replaced Oracle, Mongo replaced Oracle Coherence[1].
Ensured me to never use this for production scenarios. No matter how 'hyped' or 'convenient' it is. Yaiks.
Mongo provides no benefits of transactions or solid design. If you want either, look to Arango or PostgreSQL.
I wish someone would collect all the hyped up tech blog posts so that we always have a reference to be able to say, "see what hype does?" Sadly, I think it would only increase the number of devs saying, "Not this time, it's different."
So we followed the hype of Node.js and MongoDB because everybody was saying how it's easy to scale them compared to the established tech.
> Why do so many developers use map/reduce instead of the database for select, pluck, math?
> Why do so many developers use length instead of counting sql?
The learning curve was easy, it was very compatible with javascript, it was very easy to install, it had similar features to sql lite.
I can read their feature page but as someone who isn't an expert, I'm not sure what's marketing hype and what's genuine advantage.
Incidentally, http://www.lmfdb.org uses MongoDB (since maybe 2011) pretty heavily, as a result of me convincing them to do so after I read too much about MongoDB back then. It’s been painful for them, but it does still work well...
The mailing list is also nearly dead with only a handful of posts total during 2017: https://groups.google.com/forum/m/#!forum/rethinkdb
RethinkDB did a lot of things right from an engineering perspective, but they were too late to market to compete with MonogDB; they burned through their cash, then went out of business, and now the devs work at various other companies (e.g., Stripe). RethinkDB is not a viable choice today.
I personally put enormous effort into trying to use RethinkDB (and Cassandra, and also MongoDB); now I very happily use PostgreSQL. just like I did back in the 1999, and I’m very happy with PostgreSQL.
Since Relational Algebra was invented in 1970 we were unable to find a better representation of a relational data, there were many attempts but so far they failed (except for very specialized uses). So I'm inclined to believe it's here to stay.
JSON is basically just step backwards back to the XML Database[1] and Object Oriented Database[2]. It starts falling apart as soon as your data becomes more complex (you have relations, or need to query it by something else than primary key). Having said that, I'm glad that relational databases added support to it, it perhaps might be useful in early stages when things are in flux.
[1] https://en.wikipedia.org/wiki/XML_database
[2] https://en.wikipedia.org/wiki/Database_model#Object-oriented...
I saw a demo of MongoDB in 2010 and was thoroughly impressed. The developer experience out of the box was exquisite. During a 1 hour presentation, I was able to download the binary, connect to the shell, create records in a familiar form (JSON), query the data from my preferred language, and all within a often second-class platform, Windows.
There was a certain _joy_ to using MongoDB, but I had my doubts, too. If this form of database interaction was so easy and fun to use, what was the catch?
Well, the catch was that MongoDB was re-thinking the issue from the ground up, re-envisioning the way we work with data, in an attempt to transcend the inertia that held back existing offerings.
When I started working with MongoDB, it felt like the same transcendence I experienced moving from C++ to Python in 1998--the approach was simpler, more intuitive, and powerful.
Sure, there were some growing pains. The concept came first without acknowledged writes or even journaling. Replication came in stages. Sharding came later.
But our operation was able to go from startup-scale to a global enterprise-scale operation backed by MongoDB, performing admirably, and giving developers the ability to rapidly innovate in a low-impedance environment. We've since developed dozens of applications against MongoDB.
I've considered emergent competing offerings like PostgreSQL with its JSON store, but when I did a few years ago, they were still dramatically behind in replication and sharding support, requiring extensive administration for aspects that MongoDB handled out of the box.
And in the meantime, MongoDB has continued to innovate, bringing in pluggable storage backends, advancing the query engine and aggregation framework, adding native geo-location support, and formalizing the concept of shard zones (a key feature for global scalability). And with Atlas, they've made server administration a near zero-cost proposition.
I'm a fan because when it comes to boots-on-the-ground experience, I see nothing else in the industry that even comes close. The most pressing deficiencies have been addressed and without compromising the key innovations. As a developer and technology leader, I choose MongoDB as my default database and would only consider a traditional database if the use case was intrinsically highly constrained and tabular in nature.