Kind of an aside, but I find it odd that people treat "relational" and "SQL" as synonymous (as well as "non-relational" and "NoSQL"). You could make a relational database that was managed with a language other than SQL, right?
Kind of an aside, but I find it odd that people treat "relational" and "SQL" as synonymous (as well as "non-relational" and "NoSQL"). You could make a relational database that was managed with a language other than SQL, right?
https://en.m.wikipedia.org/wiki/QUEL_query_languages https://en.m.wikipedia.org/wiki/PostgreSQL#
I would like a better alternative, but it would need some very significant benefits compared to SQL to gain any traction. SQL is just so entrenched at this point that even NoSql database engines are adding support for pseudo-SQL query languages (which is really the worst of both worlds - the clunkyness of SQL syntax without the power of the relational model).
SELECT whatever FROM thisplace?
I know you can make it clunky with parameters and crazy stored procedures, and I’ve myself been guilty of a few recursive SQL queries that most people who aren’t intimate with SQL struggle to understand quickly, but I consider those things to be bad practice that should only be done when everything else is unavoidable.
The fact that SQL is still the preferred standard sort of speaks volumes to me about how good it is. We’re frankly approaching something similar with C styles languages. I recently did a gig as an external examiner, and it took me a while to realise that some code I was reading by a student in their PDF report was Kotlon and not TypeScript, because they look so alike.
FROM this SELECT whatever
This already allows autocomplete for the attributes to work, and has an easier mental model - you think about the tables, then you think about their attributes. It also matches relational algebra better, where you'd do the projection (picking the attributes you want) at the end.
But anyway, simple cases being simple doesn't mean the language isn't horrible for more complex ones.
One thing I always complain about is join clauses making it easy to do the wrong thing (NATURAL JOIN) and annoying to do the correct thing (joining on the defined foreign keys).
Very few people has problem with trivial queries like "SELECT x FROM y", but when query contains multiple joins or inner queries then having select at the beginning is visibly problematic.
That's (sometimes!) less performant than the one big query, but you can then refactor into a single query if you so choose.
Yeah, it's slow goings at first, but you get pretty good at SQL in the process.
In the query "SELECT foo + 2 FROM bar WHERE baz ORDER BY foo", the logical order is actually "FROM foo WHERE baz SELECT foo + 2 ORDER BY wawa" because of how each clause depends on the previous.
The SQL syntax is neither the logical order nor the direct reverse - it is just a random jumble.
I disagree. When you're reading a query you often already know what you were trying to select, you just have a problem with a clause somewhere. This suggests that the select should go last, and this makes perfect sense as Haskell's comprehensions and C#'s LINQ are both way easier to work with than SQL.
On the other hand having the select at the beginning has all kinds of problems for autocomplete, and syntactically obscures where you're selecting from and the clauses. I recommend you try LINQPad if you want experience with how much better this works:
> I've rarely ever touched filtering or aggregating lines in an exiting query unless requirements completely changed
Requirements change or bugs are discovered in the query. This is far more common than you imply.
SQL syntax is like if all arithmetic expressions had to be addition followed by subtraction followed by multiplication. And if you didn't need to add anything you would just have to add 0.
Maybe you’ve seen this thread already; a proposal with some alternatives on how to improve the situation for joining in foreign key columns, but in case not here is the link:
E.g. instead of the wrong `FROM a NATURAL JOIN b` you would use the correct `FROM a JOIN FOREIGN a.foo_fkey`, which not only needs that second name but now also loses the immediate naming of b. So e.g. autocomplete would have to look up the foreign key constraint to find out the second table. And it's still longer and harder to use than the natural join!
Most databases have one foreign key from a given table to another given table, and that simple case should be made easy to use.
https://gist.github.com/joelonsql/15b50b65ec343dce94db6249cf...
That doesn't mean that the basic design of SQL isn't awkward.
However I do also see the point of "SELECT first" just like a header you can infer the output data structure of a sub-expression without necessarily dive into the meat of it. It require a some brain training, but once you get there it oftentimes easier to navigate 100+ lines scripts by jumping from header to header (usually organized as CTE to make the code cleaner).
So does the other way around in several SQL engines.
If you write something a long the lines of select x.ID, y.NAME from bla.bla as x join hum.hum as y on x.ID = y.FK in msSQL you’ll get autocomplete on x. and y..
You’re right that it’s more intuitive to write the from first of course.
So yes, you can autocomplete
SELECT employee.Na<TAB>
to "employee.Name", but it requires you to type the table name "employee." first.
But with the from-first style you can autocomplete even bare column names - you know you have "name" (possibly even "employee.name" and "supervisor.name") and "employeeID".
* The FROM clause: First, all data sources are defined and joined
* The WHERE clause: Then, data is filtered as early as possible
* The CONNECT BY clause: Then, data is traversed iteratively or recursively, to produce new tuples
* The GROUP BY clause: Then, data is reduced to groups, possibly producing new tuples if grouping functions like ROLLUP(), CUBE(), GROUPING SETS() are used
* The HAVING clause: Then, data is filtered again
* The SELECT clause: Only now, the projection is evaluated. In case of a SELECT DISTINCT statement, data is further reduced to remove duplicates
* The UNION clause: Optionally, the above is repeated for several UNION-connected subqueries. Unless this is a UNION ALL clause, data is further reduced to remove duplicates
* The ORDER BY clause: Now, all remaining tuples are ordered
* The LIMIT clause: Then, a paginating view is created for the ordered tuples
* The FOR clause: Transformation to XML or JSON
* The FOR UPDATE clause: Finally, pessimistic locking is applied
[1] https://www.jooq.org/doc/latest/manual/sql-building/sql-stat...
2) adding new functionality requires addition of new keywords
3) you cannot define new keywords from SQL
4) despite standardization, each implementation differs
this article summarizes it pretty well, while i do not agree with everything in it it points out flaws pretty well.
There are reserved keywords and unreserved keywords. The latter can be used as table/column/function/etc names, and don’t cause any trouble.
New syntax can be invented by reusing existing reserved keywords, and introducing new unreserved keywords in places where they can’t be misinterpreted.
Not saying the problem you describe isn’t a problem, just that it’s slightly more complicated and not as bad as one might think when reading your comment.
With joins it gets more complex but still SQL could allow using foreign keys and having SELECT at the end. You could get at least something like:
"FROM Invoice i JOIN i.customer c SELECT c.name, i.number"
instead of
"SELECT c.name, i.number
FROM Invoice i
JOIN i.customer c ON i.CustomerId = c.CustomerId"
SELECT c.name, i.number
FROM Invoice i, Customer c
where i.CustomerId=c.CustomerID"
thisplace.whatever
There you go.
Compare to LINQ-syntax in C#, where you can just chain the operations however you want.
Another issue is that you can't reuse expressions. If you have an expression in a projection, you will have to repeat the same expression in filters and grouping. This leads to error-prone copy-pasting of expression or more convoluted syntax using subqueries.
lots-of-bla-bla-bla-bla as short-name
But later on you can only refer to short-name from very specific places, as you mention. So 80% of the time you're forced to go lots-of-bla-bla-bla-bla over and over and over again.
Snatching defeat from the jaws of victory.
Thankfully, most people will never have to deal with any of that, myself included. The biggest databases I've had to deal with were very relatable - one about books & authors, another about football and historic results. The other biggest database is one I'm working with and building right now, it's a DB for an installation of an application managing tons of configurations, a lot of domain specific terms. The existing database is not normalized of course, and uses a column with semicolon-separated-values as an alternative to foreign keys. Sigh. Current challenge is to implement history, so that a user can revert to previous versions. I'll probably end up implementing temporal tables in sqlite.
These days I work with millions of entries from solar production.
I’ve never had to use complex SQL more than one time.
You use tools like SSIS or APIs on top of it to get and store the data.
I know you “can” create a lot of stored procedures and views, but as I’ve already said, you really, really shouldn’t do that exactly because it’s so terrible to work with for so many people.
Honestly though, SQL with an Odata api on top of it is one of my favorite ways of storing and retrieving data. If you have to actually transform the data, you do it with SSIS or similar tools that are much more efficient top level layers that are also testable and reusable.
But to each their own I guess. The join logic never bothered me much, and that seems to be an issue for a lot of people here.
Also lack of syntax sugar doesn’t help. SELECT list could support something like “t1.* EXCEPT col1, col2”. Maybe JOIN ON foreign key would be nice. IS DISTINCT FROM for sane null comparisons looks terrible. Aliases for reusing complicated statements are really limited. Upsert syntax is painful. Window functions are so powerful that I can’t really complain about them though.
We use a lot of sql for business logic, but some code I have to reread from zero every time I need it. Maybe we modeled our data wrong or there is some inherent complexity you can’t avoid, but I mostly blame sql the language. Unfortunately I have no idea how it could be improved.
Anyway I think the sql cliff is real. Once you take a step outside the happy path prepare for a headache. For me sql definitely is in some local maxima, after all I use it every day at work.
Why don't we have SQL libraries?
I know that data models are kind of special snowflakes, but some models pop up over and over and over again and code reuse is always 0 with SQL.
To give you an example of a common problem, SLAs or the like for teams with regular business hours.
A team has to respond to a request within N hours. To calculate that I need to take into account 8 business hours per day, excluding weekends, excluding holidays (ideally localized holidays), etc.
It's a nightmare with SQL. It's precisely the kind of thing you want in a library.
Plus, obviously, standard SQL doesn't have a way to share and distribute any libraries, even if they were made. It's pre-C in terms of stuff like that.
You can model business hours and SLAs with relationships. Join on time and team.
SELECT support_request.request_time + team.response_sla AS respond_by_time
FROM support_request
JOIN team_sla
ON support_request.assigned_team = team_sla.team_id
AND DATE_PART('day', support_request.request_time) = team_sla.day_of_week
AND DATE_PART('hour', support_request.request_time) BETWEEN team_sla.start_hour AND team_sla.end_hour;Now bundle up your proposal, put it up on Github, license it as MIT, and publish it available on sqlpm.org (SQL Package Manager) so that I can re-use it.
What's that you say? I can't? There's no sqlpm.org? Not even a postgresqlpm.org?
Where's the SQL ecosystem?
Oh, wait, there isn't any because SQL is not really reusable. It's <<all>> one-off scripts, like back in the Dark Ages of software development.
What you are describing as code reuse exists for databases, but they are called applications and generally utilize a general purpose programming language. It doesn’t make sense to have a SLA data model library because every use case is different. It’s a database, not procedural code.
There's a reason monstrosities like SAP exists, they're practically what you describe.
If stuff like SAP is the future of Line Of Business (LOB) apps, instead of having a rich Open Source ecosystem of data storage and data access libraries, then we've lost.
We're locked in the trunk.
There's no reason why it can't be checked, you just need to have your schema declaration.
Name clashes are everywhere. That's why you have namespaces/packages. In SQL they are called schemas. People don't really use them these days.
The tweaking is one of the great things. If you're hardcoding queries (not sql), you're actually defining the order of operations etc. A query analyzer will use statistics, and you can hint how queries have to be executed, depending on the shape of your data.
Your code is also just strings, until you compile it. Actually, these days until you run it. Your arguments would make more sense in the 90s where people would actually compile code.
You ideal universe is actually "The Inner Platform Effect". Better let pgsql be the data platform ;-)
His arguments made sense in the 90s, and amusingly, post 2015.
Your argument made sense in the 00s and before 2015.
Swift is compiled (Apple platforms; Objective-C has always been compiled).
Rust is compiled (multiple platforms).
Typescript is compiled (so web/Javascript).
Kotlin is compiled (Android; Java has always been compiled).
C/C++ were always compiled (POSIX; Windows).
C# was always compiled (Windows; POSIX).
Almost every modern language is compiled and if it's not, it's getting a very solid static analysis step that for sure you want to have and run (PHP got types a while back, Python is getting them, Ruby is getting them).
Yeah, define the database in a database. And then define the database for the definition of the database in a database.
If you'd like to deliver a product that does something, you've got to stop adding abstraction layers at some point.
I refuse to believe that absolutely every data storage and access in this world is unique.
Everybody believe that it's unique, and that's a different story.
OS vendors also thought their hardware was special and magical a long time ago and yet POSIX was invented and suddenly they were all more or less commoditized.
I feel that we're in the teen years of data storage/data access technologies. And SQL is sort of like dental braces.
This resonates with me If I understand you correctly, with RDBMS/SQL, the structural decisions you make in the database to represent your data "poison" your application making it difficult to change over time.
What helps me is to code all of it in lower case and use something like that Datagrip with a good theme. That way you get something that is readible, colour coded and has autocomplete (very good with joins). It's the only way I've managed to keep my sanity as my experience with it grew. Bad data models doesn't reallllly impact it that much, sql is still sql even with a clean model.
I've built mini database engines in the past because of my frustrations with sql but I still use and prefer an actual rdms as opposed to trying to reinvent the wheel. There are so many features we take for granted it's not even funny. Try building your own production-ready storage system and you would quickly appreciate how deep the rabbit hole really goes.
> Good programming is about creating small, understandable, reusable pieces of logic that can be tested, given names, and organized into packages which can later be used to construct more useful pieces of logic. SQL resists this workflow. Although you can encapsulate certain repeated computations into views and functions, the syntax and support for these can vary among implementations, the notions of packages and imports are generally nonexistent, and higher-level constructions (e.g. passing a function to a function) are impossible.
The abstraction really hurts when you have to optimize slow queries and convince the optimizer to do it the right way. Entering the query plan (essentially, the annotated AST of an expression of relational algrebra) would often be helpful.
Also, SQL is ultimately text. This makes it very cumbersome to build tools that dynamically assemble queries, like ORMs or customized search dialogs, and to insert parameters. Parser performance impact overall DBMS performance quite a bit, and it would be useful to reduce overhead there too.
Haskell's comprehensions. C#'s LINQ. F#'s query providers.
SQL is really not that good actually. It had (and still has) all sorts of limitations that eventually led to new syntax, and has many quirks that require all sorts of workarounds.
For better. The only other success story just like it is JavaScript, which remains the one and only native browser programming language.
They work, they are good enough, everybody knows how to use them, gaining skills in those languages is valuable and timeless. No inane things like new languages du jour like golang that (fail at) reinventing the wheel appear every few years.
SQL remains beautifully boring & useful and is as close to program language perfection as we will ever get.
Is there something similar for SQL, where you allow an alternative syntax and maybe programming approach and then use SQL as connection to the DB ?
Javascript is a great example of exactly why this sort of thing is awful - Everyone has to use libraries like React or Vue just to make it usable, it's filled with weird backwards-compatibility junk, nobody can replace it because it is so entrenched, and attempts to make it better (typescript) end up with having to transpile backwards to javascript (rather than being able to just stand on their own).
The sooner we move to a world of web-assembly the better (but even web assembly frameworks at the moment end up having a substantial mix of javascript). We shouldn't have languages that are standard just because they are standard.
> We shouldn't have languages that are standard just because they are standard.
This is extraordinarily naive. Standards are a far more important invention than Javascript, or even the transistor. A standard existing just to have a standard is far better than every browser implementing its own scripting language. We had that once, in fact my personal website still has the text "This website is not compatible with MS Internet Explorer. Please upgrade to Chrome, Firefox, or Opera for optimal experience." even though that hasn't been true for over a decade.I obviously disagree, and think that's a pretty dismissive comment.
> A standard existing just to have a standard is far better than every browser implementing its own scripting language.
We shouldn't be stuck with a language just because one person decided to invent a language in 7 days over 26 years ago for the internet at the time, and now we are stuck with that language forever with a VERY different internet. I would like to think in 10 years time we can move to a web where people have a choice of language.
And that doesn't mean what you imply - which is that every browser has it's own scripting language - because it's possible to architect an environment which allows for multiple programming languages in the browser (see bytecode, JVM, CLI, webassembly).
Is it really that naive to think that's a better way forwards?
Apart from being subjectively displeased at SQL syntax, most problems with SQL are actually query language independent issues in the RDBMS: maddening proprietary extensions, implementation limits, nonportable details, library and system issues (e.g. character encoding and default configurations).
The best alternative query languages can do is making certain queries (advanced, rare ones) easier to express.
Working in Fox dispel many of the weird limitations of sql (and BTW, you can use SQL on Fox alike linq, is first-class).
The #1? You can build ALL using the relational model. After MS kill Fox my career can be summarized as: Trying N workarounds because I don't have Fox anymore.
Other langs (python, delphi, f#, rust) capture a little of the magic or make the workaround bearable, but none is as productive.
SQL is somewhat nice because its fairly well standardized.
This is kind of like a sticking effect. It got in at the right time and it was good enough and there wasn't enough interest in developing something to replace it that it just became standard.
But if you want to get down to it, any entity data model implemented is kind of an attempt at a replacement.
> You could make a relational database that was managed with a language other than SQL, right?
Other commenters have shown examples of non-SQL relational databases. However, all the alternatives were also examples of Structured Query Languages. Languages designed to query a data store with _Structure_.But the idea of never writing SQL again remains compelling to a certain sort of mindset, one which I have been known to share when throwing myself against the proverbial wall.
SQL isn't a bad dialect with which to hand-roll queries against relational data. I'm quite sure there's room for improvement, especially when it comes to generating queries; I've dipped my toes in this and the compiler has to 'think in SQL' eventually, it's not composable in the way that it should be.
But "SQL is annoying and ORMs are terrible" is, from my recollection, the sentiment which gave us the meteoric (pun intended!) rise of Mongo, and "relational modeling isn't negotiable" is the iron fact of our profession which lead to its ignominious fall.