Why some NoSQL DBs only let you perform transactions on a single data item

Of course, the next question is whether you actually need scaling. Outgrowing a database is a "problem of success"; if you have that problem you can throw money at the problem. It's much more important to concentrate your efforts on achieving success; choose your database focusing on your business problem, not on hopeful future potential problems.

To handle massive loads, you can trade away transactional integrity and get increased performance AND a whole new range of complicated problems that was more or less solved for you, that you now have to take care of yourself.

For some applications, the tradeoff is not a problem - and in others it requires a lot of work, up to more or less your own implementation of distributed transactions.

I have this feeling that I can't shake off , that a lot of people think relational (transactional) databases are complicated, and fail to see why they actually are complicated. This probably is not the category of people that truly need to solve their performance problems though.

But a lot of services would also get by with a distributed DB with weak integrity guarantees. Does it really matter if a few upvotes get lost on a busy post? By the time you notice sporadic integrity issues in your DB, the startup will probably have failed or raised enough money to not care.

I'm not saying it's a bad decision or that MongoDB is bad in general. I'm saying it's a decision that should not be taken lightly. If you've done the analysis and feel strongly that MongoDB's the perfect fit, then cool.

However, I think people routinely discount the operational overhead that comes with using something like MongoDB. For example, they don't think about how it's much harder to find good devops people with experience maintaining a MongoDB sharded cluster, or they don't think about what happens when the CFO wants to hook the DB up to Excel and run some queries. And I'm not even talking about the additional development complexity due to implementing integrity checks and manual transaction rollback/compensation in your application code.

Those problems can be all be solved though. It just takes more effort/money. So, you need to be sure MongoDB offers advantages to offset that.

In my opinion, schemaless data changes are not even close to offsetting the overhead, and concerns about scaling beyond MySQL or PostgreSQL are probably premature optimization. Not that those are your reasons – just reasons I often hear.

And of course, I'm making a lot of assumptions, for example that the team has experience writing & operating systems on top of an RDBMS. If the rest of the team are MongoDB wizards and you're the only one new to it, then that's a different situation.

If I was leaning towards NoSQL I would probably choose something rock solid like DynamoDB on AWS. That thing (mostly) scales by simply throwing money at it, on my last project that had tens of millions of users it worked very well, but it's definitely not cheap.

For my personal project I ended up going with PostgreSQL 9.4. The initial user experience is still as bad as I remember from years ago (after installing PostgreSQL on Ubuntu, neither the logged in user nor root have a default DB, can't even run psql, cannot create new DB's out of the box, and the tutorial(!) waxes on about "Architectural Fundamentals" instead of "Do x,y,z to install and insert a few rows of data").

With Postgres I'm able to have JSON type columns, where my unstructured documents are stored, as well as regular foreign keys, transactions and joins at the table level. So far I'm very pleased with how it's working out, I think it's a good middle ground between the flexibility of NoSQL and the reliability + querying power of SQL.

In the ideal MongoDB implementation, an "order" would be a document of its own, and all the other parts would be attributes or nested structures. So you only really need a transaction for the document itself. Other documents could be inserted/updated at the same time without creating a conflict. That's the theory, at least. Martin Fowler has a good Youtube on "Introduction to NoSQL" that I sometimes show to my students: https://youtu.be/qI_g07C_Q5I

The other thing about NoSQL is that it's often used for write-once, then read-only applications. Twitter, Facebook, or Instagram would be good examples. When 1% of your queries are INSERTs and 99% are SELECTS (with UPDATE and DELETE almost never occurring), you really don't run as much risk of anomalies. I would not recommend NoSQL for something like a banking application where you're doing lots of updates and need to guarantee consistency.

  [].slice.call(document.querySelectorAll('#post-body-2680112629272717029  *')).forEach(ele => ele.style.color = 'black')

Given that distributed transactions necessitate distributed coordination, it would seem that there is a fundamental tradeoff between scalable performance and support for distributed transactions. Indeed, many practitioners assume that this is the case. When they set out to build a scalable system, they immediately assume that they will not be able to support distributed atomic transactions without severe performance degradation.

This is in fact completely false. It is very much possible for a scalable system to support performant distributed atomic transactions.

In a recent paper [http://cs-www.cs.yale.edu/homes/dna/papers/fit.pdf], we published a new representation of the tradeoffs involved in supporting atomic transactions in scalable systems.

Do you use 2PC or something like Raft to handle distributed transactions?

From the docs it sounds like everytime a node joins/leaves the cluster goes into a (brief?) unstable mode and failures can happen, that doesn't sound great.