Serious flaws in SQL (1990)
dl.acm.org
dl.acm.org
If you really want to delete or update everything, then you should have to state it like this:
delete all rows from customers;
update all rows from items set price=0;Also fwiw DataGrip asks for confirmation before running an update or delete without a where clause
To delete/update all, add `where 1=1`
--------------------------------
p.s. In mysql I always add `limit 1` for every update. If `limit 99999` isn't enough you are probably doing it wrong.
For all rows, we typically just use `where 1=1` to encompass all rows. Allows for cleaner readability too as each clause can be separated by an AND
https://dev.mysql.com/doc/refman/8.0/en/mysql-tips.html#safe...
DELETE FROM customers WHERE TRUE;1) Write the WHERE clause first
2) Run a SELECT on that WHERE clause
3) Edit the query to make it either update or delete as required
Similarly, I only ever perform recursive filesystem commands using find and xargs. I run the find first, then hit the up arrow and pipe it to whatever I want to do (I learned that one from a very patient sysadmin after I accidentally chmod'd -R / 664)
If the query was somewhat involved to determine what to delete, I’d write the select query to just select row IDs of rows for deletion (assuming a simple primary key), then wrap that with a “delete from table where ID in (select ID from …complex query…)”.
And you don't even need special syntax, it just forces someone to write DELETE * FROM [customers] WHERE 1=1 to satisfy the need for a predicate.
DELETE * FROM tbl
DELETE * FROM tbl WHERE id > 1
Data loss and corruption risks need other forms of mitigation and they'll cover the unconstrained cases as well.
It's a nice feature that adds a little bit of comfort (though I'm still wary enough to hesitate before running SQL data modifications against production)
Which is a shame as it needn't be that way. A well designed relational calculus could avoid these abstractions that come with their own problems.
Maybe I’ve been doing it wrong my whole career because every project I’ve worked with the engineers eventually said ”Screw this leaky abstraction. Having to know SQL _and_ how the ORM works on top is too much, we’ll just put everything in an ORM.rawQuery(…)”
A poor ORM-generated query might be fast enough during testing but crashes and burns when it hits client's database. Not us, but a perfect example was the recent story discussed here[1] on optimizing an ORM-generated query to get 1200x speedup.
SQL definitely has its warts, but I don't feel like ORMs make it possible to ignore those warts at all. I'd love it if a better language like PRQL was accepted by DB engines and just compiled to SQL in the DB. Was a good blog post a couple years ago that I think highlights a lot of problems with ORMs and query builders: https://gajus.medium.com/stop-using-knex-js-and-earn-30-bf41...
We should have just built better libraries around using/targeting one flavor (SQLite, e.g.), instead of wasting time with bad libraries that transformed one bit of bad code to another bit of bad code and ran it on a database engine that wasn't even doing it right...
When you solve a problem, you solve it at its source, not five miles away with a teleprompter. If relational algebra is what we should all have been using, then relational algebra is what we should have been optimizing for, not overpriced SQL database engines.
To shamelessly steal from another comment here: https://www.scattered-thoughts.net/writing/unexplanations-re... https://www.scattered-thoughts.net/writing/unexplanations-sq...
This is not remotely true of analysts and data engineers, who probably write most of the world's SQL.
I'm not sure it's true.
I think it's horrible, it's an endless torture of the beautiful-beautiful relational algebra, but it is more popular than ever and there no way around it.
> for most of the history of sql we did not know how to translate it to relational algebra, and now that we do know most databases still don't do it.
https://www.scattered-thoughts.net/writing/unexplanations-sq...
> None of this could be expressed in the original relational algebra, and once you add it all it's not obvious we should even still be calling this an algebra, let alone granting it any mathematical mystique. I'd settle for calling it the 'sql calculus'. Or 'the algebra formerly known as relational'.
> It is still a reasonably good compiler IR though, and that's still the most useful way of thinking about it.
https://www.scattered-thoughts.net/writing/unexplanations-re...
1. SQL is not mostly used as a compiler target these days.
2. ORMs do not hide SQL nastiness.
3. Relational calculus, however well-designed, does not manage to avoid the practical problems of data management.
1. There is still a lot of criticism about SQL.
2. Users use higher level languages (e.g. "ORMs") [without involving SQL?]
3. Its flaws don't impact all that many people.
4. It is a shame [that SQL is mostly used as a compiler target] as it needn't be that way.
5. A well designed relational calculus could avoid these abstractions that come with their own problems.
To be fair, it could be that you don't know what you think about those claims, but since, as the original link points out, SQL does not have a "don't know" representation...
And rightly so, while you can write whole programs inside it, the debugger/tooling/syntax for most if not all implementations is some mix of inferior/more arcane/legitimately worse.
> 2. Users use higher level languages (e.g. "ORMs") [without involving SQL?]
In certain fields (web dev in particular) with "Enterprise" development mindsets, this is often true. ORMs make it eas(y|ier) to hide your SQL.
At least, until you screw something up and end up either rolling back (expensive) or writing various code snippets (often in SQL) to undo changes. So even in the most common case, no you end up writing SQL if you are doing anything non-trivial or making trivial mistakes.
As an aside, this more or less lines up with my experience, any time there's an additional target (SQL, Assembly, IL, make files from cmake, visual studio projects from random build system, autotools), usually the higher level tool doesn't save me from having to debug the lower level one. It usually just gets in my way and I curse the person who was "making my life easier".
Source code in a common language does an unusually good job at this, actually, its a miracle I don't usually need to bust out reversing tools or stare at processor assembly. But these are the popular, well-trodden-path tools, the fringe ones usually are not worth the trouble. ORMs are just barely worth it, IMO.
> 3. Its flaws don't impact all that many people.
Hard disagree - I posit most tools worth using are more complicated than the "easy" case, which is a dictionary - inputs and outputs. A "tool" usually has several dictionaries, or several relational tables. So after your third relation, it becomes very easy to miss-design queries (maybe the tooling has just got way better? But that's been my experience, at least).
And then, if its affecting most tools, its affecting those people who use those tools. So I'd argue the effect is actually very common, if only underappreciated ("I upgraded my PC to be 10x faster and things are fast now!" - used to be a common refrain, that's less true each year).
> 4. It is a shame [that SQL is mostly used as a compiler target] as it needn't be that way.
Agreed. Although normally in ORM situations, you'll find the complex/difficult queries hard coded (with parameters of course), so it is used (as not just a compiler target?).
> 5. A well designed relational calculus could avoid these abstractions that come with their own problems.
Maybe? Despite complaining, I'm not a database guru, however if you've been paying attention to AI at all, graph databases seem to appear to be superior, in function and form. Relational databases are good enough for many things, but I don't think they are the form most natural for data, just what we as humans like to see as reports.
This is the same line the NoSQL crowd trotted out 20 years ago, complete with "they don't need joins!" as one of the commonly listed pros.
The issue is that AI doesn't care about data integrity and consistency the way, say, a medical records database, financial database, or similar database does. Social media is largely the same way. Who cares if a random comment, post, or vote is lost? It's not ideal, but specific facts are not that important. Meanwhile, a lot of RDBMSs do store data where every fact in every field of every row is critical.
We end up right back at the same point we were at with NoSQL. These alternative data stores make sense for their special purposes, but for general-purpose data storage of objects you can define the important properties of, the RDBMS works extremely well. The RDBMS is one of the oldest and most heavily tested technologies in all of computing. It's extremely unlikely that it will end up being replaced.
Im not so sure about that. Not since Postgres switched to SQL in 1995 has there even been any notable RDBMS in use.
As Codd points out in the link, SQL is not relational. He literally invented the relational model, so he is kind of the authority.
Maybe SQL is even better, but “R” it is not.
Excellent point, however how many applications are truly critical? I didn´t say DMBS (R or otherwise) are useless - I would probably reach for one before a graph database. However I´m debating that we should all use relational algebra instead of SQL. It probably doesn´t really matter for most cases.
This is certainly true: the do not hide nastiness, they simply make dealing with the db more (much more) nasty.
I mean: ORMs are now well known to "make the easy queries slightly more easy, while making intermediate queries really hard and complex queries impossible". So for anything slightly complex you still need to break out of your ORM to... yes... plain old SQL.
I think the are of ORMs is over. It simply did not deliver.
If a book on SQL is --say-- 100 pages, a book on Hibernate is 400 pages. So much to learn just to make the easy queries slightly easier to type? Just not worth it. You still need to know SQL if you use an ORM!
I prefer jooq any day over ORMs. And dont get me started over what tools like Hasuna have to offer.
There are also some languages (forgot the names) that are SQL-done-right. Select in the back, more type safe, more logic, more in the same steps as the query gets executed. These need to be adopted by PG and MySQL and we're good to go. (IMHO)
They do, though. Some of the problems Codd speaks of are not made possible in the high level language. Depends on implementation, of course, but as a rule.
I mean, think about it: If you were building an ORM, why would you make the very same design mistakes?
That's not to say you won't make all kinds of your own design mistakes (there is no ORM that isn't full of design mistakes), but I mean why make the design mistakes that are already well studied and you know to avoid?
I'm not sure if "add" is the right framing, but I posit the reason people reach for query builders/ORMs is:
1. Because SQL is not reasonably composable.
2. Because SQL syntax is horrid.
3. Because SQL implementations are terribly inconsistent.
Adding some layer of protection above the pitfalls Codd speaks to in this paper is an additional benefit, but I don't expect that is a driving motivator; still something to address. I'm probably forgetting other things, but something that tackles those issues would go a long way in reliving any desire to add another abstraction, with its own set of problems, on top.
That said, even SQL databases usually don't run SQL. They run a virtual machine against which SQL is compiled. A more enlightening question is why there are no alternative/better query languages compiled against the same target that SQL targets since that would be strictly additive.
What I didn't expect was the amount of drama within the context of those papers. In Codd's original paper on relational theory in the 60s, he spends a chunk of time dedicated to talking about IBM as "the man". Hilarious.
I had a coworker some time in the late 90's who was reading the book "Design Patterns". We were working in Java at the time and he remarked how odd it seemed to see a book that was targeted at C++ programmers use terms like "interfaces" which were "Java concepts".
In the end, we all learned to work with SQL, and to go around its idiosyncrasies. We even learned how to put objects in tables and to force trees into rectangles.
SQL comes from Boyce and Chamberlain.
In the same way we've learned to work with C and all its memory idiosyncrasies. But let's keep that between you and me as the poor Rust crowd is going to cry when you tell them that their language is unnecessary.
The problems Codd speaks of here are exactly the source of database-related bugs I continually see out in the wild. I'd expect to see it even more frequently, but a lot of developers have started using higher level languages like 'ORMs' that add some layer of protection from those issues. It aged quite well, frankly.
It goes on to give an example of two queries which "one might reasonably expect" to be equivalent but give different results. (Personally, I disagree, but maybe I've been biased too much by SQL.)
It looks like what Codd means by "psychological features" are features that are (or are intended to) make it easier for users to express what they want to do in the language, as opposed to "logical features" which are what actually enable things to do. It's similar to the concept of "syntactic sugar", I guess. In a well-designed language, there should be a relatively small, theoretically well-understood core of logical features, and the psychological features should be straightforwardly expressible in terms of those logical features. This makes it easier to understand how the language works, which to me, does seem like an important thing for a language (it makes it easier to learn, easier to optimize, easier to add new features to without causing surprising interactions, etc.)
Codd's complaint is that SQL isn't like this, specifically with regard to the psychological feature of being able to nest queries. This was intended to be a more user-friendly alternative to using predicate logic. But instead of designing the nested queries feature as a syntactic sugar layer on top of an underlying predicate logic core, both features were implemented at the "core" level. The result is a hodgepodge language which isn't clearly based on either predicate logic or nested queries alone.
1. Most of the ways you can mess up in SQL involve failing to respect unique keys in joins, and this can be done just as easily whether or not duplicate rows are allowed.
2. Enforcing no duplicates would be very expensive.
> SQL permits duplicate rows in relations
Indeed.
One of the first things you learn is that, no matter what you're doing, all tables should have a primary key. It's simply too annoying otherwise. It's just as easy to make your own surrogate key in the case where there aren't enough natural identifiers, and anyone with any experience will do so. It's almost entirely a beginner trap.
That said, I've seen so many applications get a simple many-to-many relation wrong for representing tags or categories on an object that I do understand where Codd's complaint is coming from.
> it supports an inadequately defined kind of nesting of a query within a query;
Thanks to the efforts of ANSI and ISO, this is much improved. It should be noted that 1990 is before the introduction of the LEFT JOIN syntax in SQL-92. At this time, the only JOIN syntax was the ANSI-89 "comma" join, and if you wanted an outer join you had to rely on vendor extensions. This only took about 20 years to resolve.
The alternate IN syntax at the top of p382, for example, is now valid syntax in PostgreSQL and other RDBMSs. Further, the EXISTS operator allows this type of query to occur.
> it does not adequately support three-valued logic, let alone four-valued logic.
Additional operators have improved three-valued logic quite a bit.
I'm not sure I've seen many systems that would require or benefit from four-valued logic. I'm not entirely sure what Codd is looking for here. I think he wants MAYBE for outer joins? I don't know. Indeed, most DB designs seem to go out of their way to ensure that unless you're dealing with outer joins that you're likely to only have two-valued logic. Three-valued logic is already too complicated for a lot of people.
A row with no natural is misdesigned. Proof: what does the row _mean_ if its values don’t distinguish it uniquely in the real world?