Time to Move On: Querying Without Nulls and Bags
arxiv.org
arxiv.org
* It again ignore the best solution for nulls: Algebraic types. Once you have this much get solved for free
* The argument against bags:
The best idea of the paper is that a RDBMS in fact internally do different data structure and temporal representations that are not of concern to the user. Correct.
THEN it jumps to the conclusion that bags should not be presented to the user, despite the fact acknowledge it need to display them.
This is wrong, and the major reason that is always overlooked is that assume that 2 identical values are wrong to exist.
I can have "Jhon, Jhon", be 2 separate persons, not know at this time more to disambiguating, adding a Id do nothing to help me, and yet, this data is correct as is.
The language MUST allow me to deal with this. It will absurd to claim an array language, procedural, functional, imperative, declarative, etc can and a relational can't.
It say the lang is intentionally limited for purity, that true, is ideal in some cases, but that purity is what the engine/compiler should track, but not distort MY DATA for this.
And the part where the perf of having bags dismissed is easy to debunk: implement a dbms and profile.
P.D: Is true that set unlocks lots of benefits, and in some cases are ideal. But very fast once you hit the real world you see that need both, similar how btrees alone is not enough, and then there is hash-based indexes and many others.
it sucks, but reality is unfriendly to pure pure languages
P.D.2: And to clear misunderstanding, I agree with the points in the same way Rust say the borrow checker is ON by default, but I argue we CANT make a strict rule, in special because data is not ideological pure
if the base domain is sets we do get a lot of nice constraints, particularly with semantics around projections and if we want to support bags, its trivial to add a counter for the number of identical records as an additional column.
if we want to support sets in bag-land, we could imagine an extension which support 'row unique' on insertion. or we can use a 'unique results' post-filter. so there are paths, but its a lot less clear in a SQL context.
you insist that your data demands bags. I really prefer to work in sets and bagginess is always just a source of crappy errors for me. shouldn't we really support usable models for both regardless of the underlying storage?
Correct.
> you insist that your data demands bags.
No, I insist that real data needs both. You can bias the language/engine in one way or other.
The underlying problem is that once you say:
> if we want to support bags, its trivial to add a counter for the number of identical records as an additional column.
Then it means that this is second-class.
The main gripe I have, is that the relational model has this beautiful promise of "make the storage invisible" but need to have a coherent semantics about the data.
If you say "2 identical values/rows can't exist" then we adding the same problem that all the rdbms has: None implement the relational model, and none can deal with "relation on relations".
Imagine how crazy if I tell you a language can't do `[[1,2], [1, b10000]]` and to get into `b10000[:1] where ...` is necessary to switch to another paradigm.
P.D: note how "2 identical values can't exist" break when you have 2 columns, of numerical values, where the slice of both are the same value pair-wise: Are the 2 columns wrong?. And if both are called the same? same type? from different or same table? etc. "Duplicated" values are everywhere and is trivial to get there.
And if you "add a column" then it breaks the soundness of the language:
IN: [1, 1] != OUT: [1: 1, 2: 1]I write a lot in other language that uses ADTs, and I really miss STTs for the ease by which I can construct a "Partial" data type with significantly less boilerplate.
Of course, it also enables laziness, as instead of considerig whether it's proper to have a specific Partial product type, I just create it and move on. Easier to break loosey-goosey on modeling the domain.
There have been several languages claiming to fix SQL and none of them (to my knowledge, I'd love counter examples) have achieved widespread enough adoption to be named a proper successor.
Reasons I can guess:
- SQL's 50 years of entrenchment- all RDBMSs speak it! Most monitoring systems speak it! Any successor language needs a good interop story so people can use it with their existing systems.
- SQL is good enough for run of the mill day to day tasks- and these days by the time I'm lost in recursive queries or window functions, I can ask an LLM for help. Maybe a successor language can win on IDE support or other parts of the dev/agent experience
Successor languages also tend to only replace parts of SQL (usually the queries, not the insert/update stuff). I think PRQL does this (once again I'd love to be wrong). Now the dev has to learn two languages?
I guess my point is that a successor language, can't just fix the semantic issues with SQL, to be successful it also has to provide large ecosystem (and maybe political) steps up. I didn't see any of that in this abstract, which kills my excitement.
There is NOT other *good reason*.
P.D: But, why? Well, at first SQL was truly small. Then the cost of "just adding" something look easier than redo another lang.
TODAY, SQL is insanely bad and complex.
To the point that you can, literally, learn Rust faster than SQL.
Seriously! look for example how many reserved words you need to be aware of:
https://en.wikipedia.org/wiki/List_of_SQL_reserved_words
P.D: I implement parts of the SQL support for a RDBMS. I think the teams expect >5 years to fully do it.
I bet is possible to implement Rust way faster than that (ie: I mean here for both: Parsing and correctly executing, not all optimization and other concerns)
It’s pretty interesting!
I think its similar to regex that way. There are deficiencies that are hard to objectively defend, but at the end of the day its something i enjoy using and gets the job done.
I accept SQL as an industry standard,and i've been solely focusing on databases for th4 last decade or so, but it is such a horrible language nevertheless...
That said I've led such a migration, and there's no such thing as "standard SQL" in real practice. You're always going to need extras or optimizations.
Rel itself looks really nice, but I have a hard time imagining trying to justify a new query language in business terms. Breaking with SQL completely doesn't seem worth it compared to simply extending or restricting the language like every engine does anyway.
Rel seems to offer more than SQL, it's just hard to assess the value.
A SQL engine whose dialect enforces unique rows and doesn't include NULL isn't even so different from standard.