Which is a shame as it needn't be that way. A well designed relational calculus could avoid these abstractions that come with their own problems.
This is not remotely true of analysts and data engineers, who probably write most of the world's SQL.
I'm not sure it's true.
> for most of the history of sql we did not know how to translate it to relational algebra, and now that we do know most databases still don't do it.
https://www.scattered-thoughts.net/writing/unexplanations-sq...
> None of this could be expressed in the original relational algebra, and once you add it all it's not obvious we should even still be calling this an algebra, let alone granting it any mathematical mystique. I'd settle for calling it the 'sql calculus'. Or 'the algebra formerly known as relational'.
> It is still a reasonably good compiler IR though, and that's still the most useful way of thinking about it.
https://www.scattered-thoughts.net/writing/unexplanations-re...
1. SQL is not mostly used as a compiler target these days.
2. ORMs do not hide SQL nastiness.
3. Relational calculus, however well-designed, does not manage to avoid the practical problems of data management.
1. There is still a lot of criticism about SQL.
2. Users use higher level languages (e.g. "ORMs") [without involving SQL?]
3. Its flaws don't impact all that many people.
4. It is a shame [that SQL is mostly used as a compiler target] as it needn't be that way.
5. A well designed relational calculus could avoid these abstractions that come with their own problems.
To be fair, it could be that you don't know what you think about those claims, but since, as the original link points out, SQL does not have a "don't know" representation...
And rightly so, while you can write whole programs inside it, the debugger/tooling/syntax for most if not all implementations is some mix of inferior/more arcane/legitimately worse.
> 2. Users use higher level languages (e.g. "ORMs") [without involving SQL?]
In certain fields (web dev in particular) with "Enterprise" development mindsets, this is often true. ORMs make it eas(y|ier) to hide your SQL.
At least, until you screw something up and end up either rolling back (expensive) or writing various code snippets (often in SQL) to undo changes. So even in the most common case, no you end up writing SQL if you are doing anything non-trivial or making trivial mistakes.
As an aside, this more or less lines up with my experience, any time there's an additional target (SQL, Assembly, IL, make files from cmake, visual studio projects from random build system, autotools), usually the higher level tool doesn't save me from having to debug the lower level one. It usually just gets in my way and I curse the person who was "making my life easier".
Source code in a common language does an unusually good job at this, actually, its a miracle I don't usually need to bust out reversing tools or stare at processor assembly. But these are the popular, well-trodden-path tools, the fringe ones usually are not worth the trouble. ORMs are just barely worth it, IMO.
> 3. Its flaws don't impact all that many people.
Hard disagree - I posit most tools worth using are more complicated than the "easy" case, which is a dictionary - inputs and outputs. A "tool" usually has several dictionaries, or several relational tables. So after your third relation, it becomes very easy to miss-design queries (maybe the tooling has just got way better? But that's been my experience, at least).
And then, if its affecting most tools, its affecting those people who use those tools. So I'd argue the effect is actually very common, if only underappreciated ("I upgraded my PC to be 10x faster and things are fast now!" - used to be a common refrain, that's less true each year).
> 4. It is a shame [that SQL is mostly used as a compiler target] as it needn't be that way.
Agreed. Although normally in ORM situations, you'll find the complex/difficult queries hard coded (with parameters of course), so it is used (as not just a compiler target?).
> 5. A well designed relational calculus could avoid these abstractions that come with their own problems.
Maybe? Despite complaining, I'm not a database guru, however if you've been paying attention to AI at all, graph databases seem to appear to be superior, in function and form. Relational databases are good enough for many things, but I don't think they are the form most natural for data, just what we as humans like to see as reports.
This is the same line the NoSQL crowd trotted out 20 years ago, complete with "they don't need joins!" as one of the commonly listed pros.
The issue is that AI doesn't care about data integrity and consistency the way, say, a medical records database, financial database, or similar database does. Social media is largely the same way. Who cares if a random comment, post, or vote is lost? It's not ideal, but specific facts are not that important. Meanwhile, a lot of RDBMSs do store data where every fact in every field of every row is critical.
We end up right back at the same point we were at with NoSQL. These alternative data stores make sense for their special purposes, but for general-purpose data storage of objects you can define the important properties of, the RDBMS works extremely well. The RDBMS is one of the oldest and most heavily tested technologies in all of computing. It's extremely unlikely that it will end up being replaced.
Im not so sure about that. Not since Postgres switched to SQL in 1995 has there even been any notable RDBMS in use.
As Codd points out in the link, SQL is not relational. He literally invented the relational model, so he is kind of the authority.
Maybe SQL is even better, but “R” it is not.
Excellent point, however how many applications are truly critical? I didn´t say DMBS (R or otherwise) are useless - I would probably reach for one before a graph database. However I´m debating that we should all use relational algebra instead of SQL. It probably doesn´t really matter for most cases.
This is certainly true: the do not hide nastiness, they simply make dealing with the db more (much more) nasty.
I mean: ORMs are now well known to "make the easy queries slightly more easy, while making intermediate queries really hard and complex queries impossible". So for anything slightly complex you still need to break out of your ORM to... yes... plain old SQL.
I think the are of ORMs is over. It simply did not deliver.
If a book on SQL is --say-- 100 pages, a book on Hibernate is 400 pages. So much to learn just to make the easy queries slightly easier to type? Just not worth it. You still need to know SQL if you use an ORM!
I prefer jooq any day over ORMs. And dont get me started over what tools like Hasuna have to offer.
There are also some languages (forgot the names) that are SQL-done-right. Select in the back, more type safe, more logic, more in the same steps as the query gets executed. These need to be adopted by PG and MySQL and we're good to go. (IMHO)
They do, though. Some of the problems Codd speaks of are not made possible in the high level language. Depends on implementation, of course, but as a rule.
I mean, think about it: If you were building an ORM, why would you make the very same design mistakes?
That's not to say you won't make all kinds of your own design mistakes (there is no ORM that isn't full of design mistakes), but I mean why make the design mistakes that are already well studied and you know to avoid?
Maybe I’ve been doing it wrong my whole career because every project I’ve worked with the engineers eventually said ”Screw this leaky abstraction. Having to know SQL _and_ how the ORM works on top is too much, we’ll just put everything in an ORM.rawQuery(…)”
A poor ORM-generated query might be fast enough during testing but crashes and burns when it hits client's database. Not us, but a perfect example was the recent story discussed here[1] on optimizing an ORM-generated query to get 1200x speedup.
SQL definitely has its warts, but I don't feel like ORMs make it possible to ignore those warts at all. I'd love it if a better language like PRQL was accepted by DB engines and just compiled to SQL in the DB. Was a good blog post a couple years ago that I think highlights a lot of problems with ORMs and query builders: https://gajus.medium.com/stop-using-knex-js-and-earn-30-bf41...
We should have just built better libraries around using/targeting one flavor (SQLite, e.g.), instead of wasting time with bad libraries that transformed one bit of bad code to another bit of bad code and ran it on a database engine that wasn't even doing it right...
When you solve a problem, you solve it at its source, not five miles away with a teleprompter. If relational algebra is what we should all have been using, then relational algebra is what we should have been optimizing for, not overpriced SQL database engines.
To shamelessly steal from another comment here: https://www.scattered-thoughts.net/writing/unexplanations-re... https://www.scattered-thoughts.net/writing/unexplanations-sq...
That said, even SQL databases usually don't run SQL. They run a virtual machine against which SQL is compiled. A more enlightening question is why there are no alternative/better query languages compiled against the same target that SQL targets since that would be strictly additive.
I think it's horrible, it's an endless torture of the beautiful-beautiful relational algebra, but it is more popular than ever and there no way around it.
I'm not sure if "add" is the right framing, but I posit the reason people reach for query builders/ORMs is:
1. Because SQL is not reasonably composable.
2. Because SQL syntax is horrid.
3. Because SQL implementations are terribly inconsistent.
Adding some layer of protection above the pitfalls Codd speaks to in this paper is an additional benefit, but I don't expect that is a driving motivator; still something to address. I'm probably forgetting other things, but something that tackles those issues would go a long way in reliving any desire to add another abstraction, with its own set of problems, on top.