How did SQL get so popular?
therealadam.com
therealadam.com
1. modern programming frameworks all abstracted away SQL because people found it hard to understand
2. SQL written by ORM layers in these frameworks is slow. So people decided this was because RDBMS are slow.
3. So people decided to stop using RDBMS and use object-oriented data stores instead (which is faster, unless you have relational queries, which a large number of simple web applications do not)
4. Then when they started needing relational queries again, they started building complicated custom layers on top of their object stores to handle this
I figure sometime in the future somebody will publish a "brilliant" article called "Save yourself thousands of lines of object-manipulation code... by using RDBMS!" and suddenly SQL will be "popular" again.
I'm the first to admit that the default approach to web development for the past 10 years has been "slap all your data into mySQL, then start writing code". It works in a very, very large number of cases, but it's not super-efficient. It never claimed to be. It was just really easy and quick to write.
Now tools are growing up that let you solve these sorts of problems without using RDBMS, and that's great: the huge number of apps that don't need RDBMS can stop using SQL, and let us SQL junkies get on with our lives :-)
Web developers on these types of projects cannot either a: buy a "real" SQLDBMS, or b: charge their customers so their business model makes sense without a huge number of transactions. This means they are willing to suffer it out with solutions that don't provide flexibility or data correctness guarantees of a RDBMS.
This is just a guess, but it's quite likely that it is the rise of hyper-scale-intensive, speed-intensive, revenue-light business models that drives devs towards deciding SQL isn't worth it.
I'm not going to claim it's perfect, but it's pretty damn good, and postgres + intelligently used caching (such as memcached) goes a long way.
My code was hard to understand (and very hard to write, at times) but the result was a much lighter, much faster application.
No. Not because it's complicated. Because they wanted higher level facilities for getting/setting data. In other words, they put in abstraction because abstraction is good, not because they are dumb.
The people who made the frameworks would require a good understand as well as the idea that an abstraction might make life easier.
The side effect is that if you create a good enough abstraction you'll get people who can handle the abstraction but are a little shakey when dealing with what is abstracted.
See, you can both be right :)
The reason, say, C++ is a good abstraction for machine code is because it simplifies what's going on to the point that you can write something very complex in fewer lines of code, allowing it to fit into the human brain better. I'm willing to be shown examples to the contrary, but ORM layers don't simplify or compress relational queries, they just change the syntax to something more "object-y".
So maybe my point is that abstraction is good, but ORM isn't really abstraction.
Microsoft's Linq for SQL generates mostly OK SQL from your code. While not perfect for all complex operations (like multiple joins with aggregations), the SQL isn't horrible. For most use cases though, the SQL seems to be on par with what I would write myself. Ofcourse this is a rather vendor-specific implementation (for Microsoft SQL Server), but there are decent ORMs out there.
The biggest problem really comes when people don't understand what underlying SQL will be required for specific operations and just write code blindly. This can cause things like the famous nested-loops effect instead of efficient joins, but this is solvable trough knowing how to properly use your tool.
That being said ORMs are good at poking at small amounts of data (single or few objects), while SQL is probably the best language there is for huge set-based operations. When you are doing set-based things, not using SQL will most likely be a bad choice.
I suspect that a big part of the problem with SQL is that it's too abstract: it is both hard to predict how fast a query will run and hard to predict whether the query will continue to run fast as the database changes. A big part of the appeal of things like memcached and Redis is that their performance is not only fast, but also predictable.
No, it's a higher level facility for querying data when you want different result sets back randomly, it sucks most common operations of inserting or updating data.
Forcing me to write a very long statement to insert a row isn't in any way more abstract that saying anObject.save().
SQL continually forces one to write the same boring code over and over. As soon as you start automating the generation of the boring bits, you've started to build an ORM. ORMs exist because programmers get tired of writing the same code again and again.
While SQL is good at what it does, it doesn't do the thing programmers need done most, which is store data structures easily without making them write a bunch of mapping code to translate it first. SQL isn't confusing, it's just frustratingly boring and repetitive.
I continue to be surprised by the number of hackers who don't know SQL well. And when developers say you can only really benefit with RDBMS when you have petabytes of data, I wonder if they haven't worked on apps that are explicitly data-centric. Granted, I spend my work days in apps that are or closely resemble data warehouses, but I can get your standard RDBMS engine to do a tremendous amount of work before I have to touch the first line of procedural code. If it fits your needs, by all means jump feet first into the RDBMS of your choice. When it doesn't fit, know enough to realize it and make a considered decision (seldo, +1 for mentioning the huge number of apps that don't need RDBMS)
The point I'm really trying to express is that any hacker that touches a database would probably be well served to spend some minimal amount of effort learning ANSI SQL and understanding basic indexing, basic normal forms, and maybe star schemas. Adding some intermediate SQL and RDBMS skills to your toolbox is just another step towards maximizing your effectiveness as a developer.
It sounds like it never even crossed his mind that the relational model is a good fit for some problems. Wow.
98% of programmers think the word "relation" has something to do with links between tables. I doubt the OP has ever even heard of a concept called "the relational model".
I was expecting to see hierarchical databases, OODBs and alternate query languages mentioned. The claim for "Distaste" for SQL is odd and vague. I agree with the first point - "Relational databases are ACID and that's powerful" but I ask "so what, where's the distaste?"
People who know a bit about RDBMSs know that they do a lot of heavy lifting for you. Is the point that a younger generation don't know that? What are they teaching kids in college these days?
Massively parallel data storage systems are in vogue now, bacause of the very the few very popular web apps that need to scale a lot horizontally, and SQL databases don't really deal with that. But that's not "How SQL got popular". Maybe it's why some have forgotten.
They have a terrible experience using MySQL on a database with one huge, un-indexed table, and they're convinced for life that all RDBMSs are crap. I mean, if you don't like relational databases for a real reason, that's one thing, but this is like complaining because gcc won't run a Visual Basic program you copy-and-pasted, and having half the people you whine to not know any better, so they tell their friends too.
I can understand when people complain that SQL is a lousy implementation of the relational model (or about its syntax being a bit verbose, etc.), but that's clearly not the case here.
The prime time for SQL has long been over, it is a necessary tool most of us don't think too much about.
I have to respectfully disagree. Its often when a tool becomes so ubiquitious and sufficiently intuitive that its true power comes to the fore.
It is like the electric grid, when it was first being forged everyone who used it thought about it extensively, now hardly anyone does. But that does not mean the grid's prime time is over, it means that now it is truly becoming powerful.
Every time a framework pulls a big set of data across the wire to do a local calculation on it (a for loop summing a list of values, for example), it's doing a completely unnecessary round trip to do (probably in PHP or Ruby, no less) a task for which the database has already been specifically optimized. It could have done all the calculation during the same pass it took to select the records and just sent the answer instead.
Frameworks obscure this.
Just a thought.
This is not what relational databases were designed for.
Otherwise your comment is correct.
Interesting data point, by the way. I have a little USB flash drive that stores 4 GB. It was about $10. That's probably more disk storage than existed in the entire world until Y years ago -- any idea what year?
The "distaste" for SQL comes (as far as I can tell) from attempts to work around it. If you just want to write a Python program, SQL becomes just a bunch of boilerplate that you want to avoid. If you want software that does X, and you know how to use your database, it will help you enormously.
Relational databases became popular because the declarative high level query language was less time consuming to write queries in than earlier non-relational database.
SQL is far from the perfect query language though. It's syntax is often bizarre and confounding, and has many non relational extensions (column ordering, duplicate tuples in a relation).
The relational model is great, and it's important to remember that the things that make it great are that it is simple, dynamic, easy to understand and helps ensure you get correct results. IMO the two most common mistakes people make are being overly worried about breaking the model where necessary, and conversely, not realizing the benefits of things like human readable keys.
Missing is the concept that SQL is really quite powerful.