I have yet to see a real use case for Mongo unless you are building a Facebook clone. Can someone suggest when it is actually useful over a properly tuned relational database?
I guess I kind of reached the irrelevance stage just thinking about the sort of problems it would be suited to rather than actually building anything.
People seem to keep making the mistake of building a social network in mongo. It's entirely unsuitable for that.
Source: I've helped 3 companies move their social networks from MongoDB to OrientDB when they figured out that MongoDB prevents them from shipping features that users expect, e.g. friend-of-a-friend style queries
What server language were you using and where were you deploying?
I've been using node.js (I develop the official driver - https://github.com/codemix/oriento), but there's pretty good libraries emerging for other languages too. Most people I've worked with are deploying to AWS, one company was running on the bare metal.
Do you have any advise for someone thinking of deploying on GCE?
My use-case would be for an online code editor (Plunker, if you've heard of it) with users, projects, packages, collections (of projects), comments and project versions (stored as content-addressable git-compatible objects).
I'm also interested in understanding if there is any built-in compression mechanism because I will be storing a large volume of very similar text files. Any hints?
Have you used the lucene indexes much? If so, can you do any of the crazy faceting delivered by ElasticSearch?
No, sorry, I've never used that. However, generally - like all dbs, Orient is happiest when it has access to a lot of RAM. Also the Write Ahead Log can take up a significant amount of disk space, those are two things to be immediately aware of.
> I'm also interested in understanding if there is any built-in compression mechanism because I will be storing a large volume of very similar text files. Any hints?
Sadly it doesn't yet do leveldb style document compression, and I've seen no hints that it's on the horizon, but the OrientDB guys are pretty responsive and would probably be open to the suggestion.
> Have you used the lucene indexes much? If so, can you do any of the crazy faceting delivered by ElasticSearch?
I'm just starting to use the lucene indexes now so I can't give much feedback on those yet. It should be possible to do faceting using SQL to a limited degree, but I don't think there's native support for it yet. I think that will get improved in the next few versions because people are crying out for it.
I don't want to create a schema in a relational database for that kind of use case. A MongoDB fits as well.
Edit: Sorry for my short and unprecise comment. I regret posting it, loosing all my Karma. Yes, of course I would use a schema, but I may use hundreds to get my products presented with all attributes and variants available. There is a set of attributes that will remain for all products, others will vary, some will change and will be dropped. Some may be conditional, depending on size or color. I could model that in RDBMS as well, adding those properties in an additional JSON or something else.
Yes, I'm also mainly using relational databases and just wanted to give an example of a use-case for MongoDB. I regret, sorry.
@see: http://docs.mongodb.org/ecosystem/use-cases/product-catalog/
I personally miss Joins in MongoDB - but other vendors support Joins.
Now things become more hairy when you realize that perhaps you want to keep more information about the actor so for example you want to change details.actor to details.actor.name. You will have two choices, either run through your database and modify all documents (which is something similar to RDBMS is doing) or write code to handle both cases. The second one seems easier, but as you'll have more changes it'll come back and bite you hard.
Later in the future you might realize that by repeating details about actor for every single movie you simply not only wasting a lot of resources (your database is bigger and slower) but also this affects integrity (in one movie perhaps you include actor's middle name, or maybe you have a typo).
At that point you'll start creating a collection that holds actors and then only store key of it in "details.actors". You will soon realize that you're basically reimplementing a relational database on top of Mongo, except not only it is way slower, your application is becoming more complex.
There are uses for NoSQL, but whenever you have to ask yourself which model you should go with pretty much always the answer will be: relational.
You are always going to end up creating a schema, whether it's explicit in your tables or implicit within your code. Otherwise you end up with completely heterogeneous data which is impossible to query in any useful way.
This is a red herring.
In a sql db, every single thing needs to fit into a flat schema with nesting done by relations. Additionally, it's very rigid. Maybe you just don't want to set up a laces table, and 4000 other tables for every little attribute of every product. Of course this is not how you would do it in real life, there are better ways. Mongo is one of those ways.
Saying that Mongo is good for nothing is just as dumb as saying that Mongo is good for everything. In 2015, the Mongo backlash is just as tired as the Mongo hype.
Nope, see Postgres (hstore and json).
Though i suppose it gets trickier if attributes may have sub attributes etc....
If you say so.
Diaspora is a social network that made the mistake of choosing mongodb for their first iteration. Here's a blog post where one of the involved developers explains why it's not good for that, even: http://www.sarahmei.com/blog/2013/11/11/why-you-should-never...
Seamless auto-sharding, supported as a first-class use case. Particularly in a cloud environment where you can have your system automatically spin up more hosts as load increases (and take them down as load drops) with zero human intervention, which is really nice.
There are plenty of alternatives in that space nowadays (more than when mongo first came out), and reasonable people can disagree over whether to use mongo rather than e.g. cassandra, riak, or shonky third-party clustering addons for mysql/postgresql, but I've yet to see an affordable relational database with out-of-the-box auto-sharding support that comes anywhere close to what mongo offers.
(Also, first-class support for async-io clients. This is about client libraries rather than the server itself, and purely an artefact of when mongo was released, but if you're calling e.g. PostgreSQL from the JVM, most of the libraries are oriented towards blocking JDBC which is "good enough" (there are valiant efforts like https://github.com/mauricio/postgresql-async but you can't use them with established higher-level libraries). Whereas every mongo library offers a callback- or future-based API, and the higher-level libraries are built around this)
FoundationDB is a bit weird, but it handles automatic scaling-out, and has an ANSI SQL interface (backed onto a K-V store): https://foundationdb.com/key-value-store/white-papers/scalab...
Caveat: I've not used it for more than hobby stuff yet, but it's a fascinating bit of technology!
That said, I have found one specific use-case where it has incredibly worked well, which is as a JSON-based file system in which I can query into the document structure to find the records and data I need en-masse with very simple queries. A project I've been working on for a few years has its own custom schema that fits far better into a JSON hierarchy than a relational one. In this specific system there are very few records (compared to our primary RDBMS which has millions), and when not being explicitly maintained, the files are generally static. The single-collection database in question with a few hundred JSON objects would likely have to be split into at least 10 tables with thousands of records each.
When it comes to using said "files" in production, I index the objects into memory for quick retrieval. In order to populate said cache, MongoDB allows me to query into the JSON file structure and retrieve the necessary "files" very simply into however many indices I need. And that's where it has shined in my experience; Not as a live read-write datastore for production use, but as read-heavy storage for a relatively small amount of JSON data with a deep hierarchy that would require an annoying amount of joins and subqueries for otherwise simple queries.
I've also tried it out with our multi-system logging system using capped collections (JSON formatted messages transported and collected with rsyslog). But we grew beyond its limits almost immediately and ended up in far more manageable territory with kibana / elasticsearch.
Why does a traditional relational db fail for such a well-defined data model? I mean, there's nothing about 'given an song (id), retrieve artist and album' that would make mongo inherently better.
Maybe I'm getting old, but I really want DB to just work (unlike HBase, at least a few years ago). Other than that, I'm flexible.
Use mongoose!
>On the other hand, I do miss JOINs when I need them (though we solved that on app level).
How you do that? (maybe is different than me)
What about transactions?
We don't use node.js. Also, we have app-level schema, but enforcing should (IMHO) be done on DB level so as to avoid any chance of invalid data.
Joins: easy, we specify which fields are connected (on app level), then fetching goes to connected collections and fetches data from there too. Not ideal, but it works. We made a similar solution for foreign keys.
Transactions: no need for any further guarantees. Our system mostly works on a single record at a time in all critical components. If this is not possible, we have app-level locking to avoid conflicts. No issues or wishes here.
app-level locking is a recipe for disaster if data matters and more than 1 process/user/client/etc can access your data. if the data doesn't matter then I guess you can do whatever you want.
web "developers" have a bad reputation for a reason. so many amateurs amongst web developers...
As for app-level locking, I am talking about sacrificing performance, not safety. We just make sure that some piece of code runs exactly once. Since the need for this is very rare and the places are not performance critical, we can live with that. So no, we have no need for additional transactional guarantees on DB level.
It's most definitely directed at you.
> You are a Java dev I presume?
C#/C++. Though I've worked with java before amongst others.
> As for app-level locking, I am talking about sacrificing performance, not safety.
You are sacrificing both. Only idiots truly depend on app-level locking.
> We just make sure that some piece of code runs exactly once.
Amateur hour...
> Since the need for this is very rare and the places are not performance critical, we can live with that. So no, we have no need for additional transactional guarantees on DB level.
So it's a useless pointless trivial application...
Another day, another idiot, I guess. (and yes, the term is definitely directed at you - if you still follow this throwaway account anyway)
I've been building https://rwt.to and https://movingjoburg.co.za with storage backed by MongoDB. Both are transit apps/websites in South Africa, could be more useful, except that we spend most of our time collecting and capturing data.
At the time I was very inexperienced (I guess I'm a bit better now?), but these are the reasons why I chose MongoDB.
1. Geospatial query support out of the box (this is during 2.0-2.2). It's now great with GeoJSON, I store routes and stations as GeoJSON, and querying them is very easy. Compare with PostgreSQL + PostGIS.
2. Some bus services have weird schedule structures, and I needed to be able to generate those schedules, I store schedules as a giant JSON file, and I can query that file to find when the next bus/train is. It's now possible with JSONB in PostgreSQL, but trying to fit that into a relational model wasn't worth the pain at the time, especially when there's always edge cases that I have to cater for at times.
3. To create a pricing engine on rwt.to, I needed a very flexible schema, because I have to cater for many cases (are there bus/train transfers, discounts, and quite different rules for those transfers and discounts). MongoDB gave me the ability to vary my schema in the instances where I needed to store different types of data. I'd rather contend with that than have a table with 100 columns to do the same. To head off on a tangent, in GTFS this data (fares) is computed and stored in the table, but I calculate fares at query-time because there's sometimes a lot of rules to consider, making calculating them and caching them like GTFS unpleasant.
4. Other reasons was because it was pleasant to work with MongoDB + Mongoose.js. This is very important as I'm the sole developer, and due to my line of work, I don't get all the time I need to work on both projects. For my core transit data, I won't reach >20GB even when I manage to collect all the data for South Africa, so I won't have to contend with the 'big data' issues that other users would face.
PS: Out of interest, if anyone would like to try rwt.to to see how it works, here are some example links: - https://rwt.to/*mrwF8w0u - https://rwt.to/*mrCLEZaE - https://rwt.to/*mrOkEDKH
In your own question you kind of hint at it. You need a tuned database. NoSQL lowers the barrier to entry for fast persistent data storage with replication. NoSQL doesn't replace SQL databases, it's looking to optimise a use case where transactions are not required.
Personally I use Mongo a bit like a cache, sitting in front of a SQL DB. Things that need to be ACID are handed back and forth. Those that don't are kept in Mongo and I get the best of both worlds.
I'd be curious to know in what contexts anyone would not want transactions.
I readily see the case for memcached or equivalent when it comes to caching -- it is very valid, and indeed useful, when slightly outdated data is perfectly acceptable. I can even picture how you might be using MongoDB to do the same, even if you admittedly have me wondering how you're invalidating the mess.
For a persistent data store, however, I'm honestly at a loss. In my own experience, transactions are needed as soon as there's a remote possibility that a concurrent write may occur. Even embedded systems need them, when you're threading statements concurrently for performance reasons, or when you're subsequently merging local data with another node. (See CoreData/iCloud bugs for what occurs when you ignore ACID in the latter case.)
Maybe when developers don't want to bother to learn SQL?
On my CS degree we got proper introduction to SQL and the relational algebra behind it, so for me it is just a tool, not a scary monster.
I really love the data integrity options I have at my disposal with relational databases.
So for me this all NoSQL fad never made much sense. Then again, I never had to deal with a Facebook like scale problem, only daily reports from mobile network operators across their whole network elements.
NoSQL means model your data in a different way, the way you'll be using it. For some it makes more sense, for some may not.
For example, compare implementation of a tagging system in mongo and any other RDBMS, which one is easier?
As I said, I never bothered with NoSQL so I cannot properly answer it.
I've been using Mongo happily to provide online analytics solutions, but the advantage is mostly from the development side, not really from the performance side.
On the other hand, this kind of approach is great for attribute matching, which is usually a nightmare to do properly with a RDBMS.
MongoDB is easy and was used by web "developers" too lazy to learn SQL and RDBMs.
MongoDB is fine if you don't really care about the data. If you don't need ACID compliance and can afford to lose transactions.
[1]: http://www.postgresql.org/docs/9.4/static/datatype-json.html
https://docs.oracle.com/database/121/ADXDB/json.htm#ADXDB624...
Relational databases place such emphasis on reliability. Mongo is generally reliable but only two or three nines, and it's simple.
For example, if you're collecting sub-cent line items for invoices, you might prefer to collect 99.x% of the line items simply over 100% at greater complication and expense, particularly if you can measure x.
I even know someone who stores invoice items in memcached. Whenever memchached evicts something that hasn't made it onwards to real storage, they lose the ability to invoice the right customer for an ad impression.
Consider two storage solutions used to gather line items to invoices. The line items are ten cents on average.
One system is properly ACID, and running it costs one cent per line item on the invoice. The other loses 1% of writes randomly, but its hardware/backup/ops requirements are lower, so it costs just 0.1c/item.
The ACID way gives you complete invoices, but you spend 10% of the invoiced amount on the invoices. The lossy way makes your invoices 1% smaller, randomly, but the loss+cost adds up to about 2% instead of 10%.
It's like returns on physical goods, really. A random percentage of customers will return goods, you can't control that but you can estimate and monitor it, and optimize if the numbers aren't good enough.
I don't understand how you can say any of this seriously.
Overinvoicing would be bad. But underinvoicing is just another cost of business. If the cost is smaller than the alternative, fine.