SQL Databases Don't Scale
adam.heroku.com
adam.heroku.com
It's not that SQL databases do not scale per se, they just can't arbitrarily scale.
Most applications today manage pretty well with nothing more than sharding and replication. Sure, it's not the optimal solution, and it's not easy - but it gets the job done.
Facebook, at the very core (behind all the layers of cache), run, arguably, the largest MySQL landscape in the world - and they seem to be doing pretty good.
Until Heroku comes up with a better solution (or a better question), TFA is no more than flamebait.
Tell that to Clustrix, Vertica, and VoltDB.
SQL is a query language, Relational is a data model, neither one prevents scaling out.
Claiming that SQL databases don't scale, especially using MySQL as the example, is simply ignorant. Oracle, DB2, SQL Server all scale to handle databases orders of magnitude bigger than anything the most successful RoR app is running.
CICS and TPF on z/OS?
Sure code still needs to be run on top of it for specific reasons, but I submit for consideration this is true of SQL databaes, and the data persistence layer of any NoSQL based app.
Similarly CICS may have a relational DB in the loop, or it may be pointing at some sort of TPF based system, or it could be pointing at a simple pile of COBOL records stored in z/VSE. The point is CICS is the transaction manager and provides the interface to the data much the same way an sql layer over a relational DB does.
This gets especially critical once you stop assuming that data can be centralised. Once your required transactional context needs to scale large areas (geographically or logically) then the overhead of this is mighty. 2PC itself does not scale indefinitely, and while interesting research is being done in alternative transactional approaches, it would be tough to argue that any are acceptable solutions to the problem.
Also your example of banks is perhaps not an ideal one. While banks may indeed run some very large databases, they don't run one HUGE database which is expected to be always consistent. Things like ATM systems etc. run based on eventual consistency and tolerance favouring availability precisely because a simple horespower approach to a transactional database (whether SQL or not) wouldn't currently work in this domain.
My point is that there are plenty of real-world examples of SQL databases that are scaled way beyond any RoR web app's needs, so flatly claiming they don't scale is wrong.
The solution banks (and other large organizations) have arrived at -- using multiple databases with different transactional and availability requirements -- IS a solution to scaling large databases. There's no requirement that SQL databases be monolithic. Even the original article describes using a MySQL master for updates and one or more slaves for reads. That's a valid scalability solution if your application can live with the possibility of stale data in the slaves.
SQL and the Relational data model are not limited by Brewers Theorem, this is a myth.
More info here: http://sergeitsar.blogspot.com/2011/02/mongodb-vs-clustrix-c...
Our app does double entry accounting, and considering the relational structure of accounting ledgers as well as that every entry needs to be posted in an ACID manner, we don't feel comfortable building out, say, a banking system in NoSQL.
Yes, SQL is painful to scale, but it can be manageable. We partition it in a scoped schema way (e.g each company account/tenant has its own set of tables on one of the shards with the server of the fewest consumed resources, or if it's a tie, a coin toss). Scaling it in an arbitrary manner such as by names beginning A to M, N to Q, etc simply seems like a bad way to do it. If your app is like a 37 Signals app where it doesn't typically require access to another account's data, this might be a good way to do it.
One of the programmers I hired was dead set on trying to use CouchDB for everything. I told him don't try to use it for everything--it's good to use for the audit trail and document revisions on invoices, orders, etc, but not on actual double entry journals.
In my experience I have seen more atrocities committed due to developers using a relational database as a file-system, an XML document, a key value store or some variation in between.
When people complain about scaling in relational databases they usually mean that it is more rigid, and not easily adapted to changes. Which is valid and is why you cannot apply software development practices to database design. With database design you have to plan for the future and think out how your solution will grow as to not paint yourself into a corner. Further in relational design there is no such thing as premature optimization, if one thinks there is, one has already failed at setting up a robust relational database architecture.
At one of my start-ups we supported massive amounts of traffic from Hotels.com, Orbitz, Travelocity and Expedia all looking for pricing and allotment each time someone hit their front door and we did it all on applications that where backed by relational databases. We never saw Google or Facebook size traffic, but we had constant load of more than any one of those travel sites and many times load of all of them combined.
A solid application architecture and a sound relational model can scale quite well you just have to have good people that understand each discipline.
Vertical scaling of a DB is definitely an option for many people and has been used to scale many applications. However, the cost curve associated with buying bigger and bigger hardware is super-linear; doubling CPU & Memory in a single system leads to more than doubling the hardware cost. This can be problematic for many businesses whose revenue growth is exceeded by cost growth of the database.
Sharding is also an option for scaling, leveraged to great success by Facebook, Yahoo, and many others. However as the article points out, sharding prevents the developer from using many of the features that make a relational database a productive development environment. There are lots of foot guns that emerge in a sharded SQL environment and if you have not set up your development constraints appropriately, you can slow the pace of development considerably. This again leads to a cost problem because the incremental costs of adding features grows as you add more things like sharding around your database.
SQL is not useless and not hopeless. In a large number of cases, SQL is the right solution. However the techniques used to scale SQL tend to be options only to very large budget organisations. NoSQL solutions tend to be more cost effective in their scaling approach (scale out vs. scale up) without crippling the developers productivity. For these reasons, NoSQL solutions tend to be the better choice for the cost-conscious.
What Adam means to say is "some of the semantics defined by SQL are hard to get correct while maintaining scalability." Hard is very different from impossible, and there is a large number of very smart people currently solving this problem very well. There is another large number of smart people trying to solve the problem by ignoring the hard parts of the semantics. It seems they will likely come up with something very fast and scalable, but ultimately less useful for certain things which are done easier with proper SQL semantics. In fact, it's not clear in all cases that they are even making something that's more performant: cf. http://sergeitsar.blogspot.com/2011/01/mongodb-vs-clustrix-c...
Saying "SQL Databases Don't Scale" is like saying "oil paintings on wood aren't appealing". Not all oil-on-wood paintings are good, but some are, and some tempera-on-fresco paintings suck too. The logic is simply invalid.
I take issue with the assumption here. What you want, which is obviously "free/open mysql-ish thing that scales indefinatly", is not a problem people have been working that hard to solve until quite recently.
To put it in perspective, 20 years ago, many banks offered no access to your money outside open hours and the internet was not a thing normal people used.
Today, if you go to Oracle or IBM, you'll find that they'll be happy to help you solve your enourmous problem at great profit to them. The thing that's changed in the last few years, is that Web 2.0 guys want the same power (or more) for a tiny fraction of the cost.
This is a good thing. This is exciting. People recognize that the status quo sucks and are working hard on change. The solution will probably involve some SQL, and probably some other tools as well. Don't be such a downer. ;-)
It's also hellishly complicated, but so is the problem.
Then, just another billion on transition costs, right? :)
NoSQL databases scale much easier. Just add a new node and you are done. +1. Many of them are also freeware. +1 again. Many of them are also faster, because no parsing is done, +1. Many of them don't have and don't need locking. +1. No looks mean no write waiting. +1.
Decide yourself.
I'm not trying to defend the article though.