I don't want to learn your query language
erikbern.com
erikbern.com
Lucene for example doesn't use SQL because its really solving a different problem - text search. Its a language dedicated to what could only be expressed using something like `LIKE` and regexes in SQL.
Same with Splunk, it addresses a different domain and solves different problems that cant be easily expressed in SQL.
Same with MongoDB.. its not a relational db. Just because there is a mapping from some SQL queries to some Mongo queries, does not mean they are identical databases.
The query language directly impacts the AST generated, which is necessarily tied very closely to the exact capabilities/internals of a system. This post just feels like its based on a cursory glance of what the systems do.
If all Lucene had was regexes, it would not perform much better than a SQL database throwing regexes at strings. Its precisely because the query language is finer-grained that it can be better optimized for that usecase.
But for other domains, the query languages arose out of need, not because someone felt like it. SQL is hot garbage for for non-trivial logic. I can't even imagine what splunk would look like if you had to query it with SQL.
If anything, SPL (or rather the piped operators approach) would make a lot more sense as the groundwork for a universal query language that works across many domains.
There are much better options than LIKE and regexes in SQL for text search. To mention but two:
SELECT 'a fat cat sat on a mat and ate a fat rat'::tsvector @@ 'cat & rat'::tsquery;
https://www.postgresql.org/docs/current/static/textsearch-in... SELECT * FROM articles
WHERE MATCH (title,body)
AGAINST ('database' IN NATURAL LANGUAGE MODE);
https://dev.mysql.com/doc/refman/8.0/en/fulltext-natural-lan...All main engines support full text search in one way or another. TSQL, Oracle, DB2... even SQLite.
What more, these engines all support inverted tree indexes in one form or another - i.e. the same type of index Lucene uses - to make this efficient. This is completely different from the typical LIKE or regex query against a column with a BTree index.
Also:
> Squeezing the query language into some sql-like shape will provide marginal gain at best.
... is disingenuous at best. In both examples I raised (and in the ones you'll find if you query the others) the engines have functionality to accept user input pretty much as is - meaning as your typical mom would type it in a search field.
If you need extra criteria from there and the dialect's full text syntactic features, you can use regular SQL conditions (and joins, and aggregates, etc.) on the subset of records found. The most discriminating criteria/index will be the one related to the search query in most cases. And when not, lucky you - your query planner did not hammer everything with the same sledgehammer, thus demonstrating why you should be using SQL.
What I see at your zombodb link is just a way to make Postgres proxy requests to ES - complete with ES syntax and/or Query DSL. The only difference is that instead of connecting to ES directly you connect to Postgres and then add "SELECT" thing in front of your ES Query DSL. If it makes something easier, fine, but it's not "Postgres full text over ES".
Think of it as browser prefixes. Sooner or later all engines agree on a common syntax, with each one needing to support some legacy code.
And in contrast with what happens when you use some third party app, you can further filter matched rows, join them with other tables to your heart's contents, aggregate them, apply window functions, and what have you.
Take a look at the most recent versions of the SQL standard, say SQL:2016, and you see it handles document type data (JSON) pretty well too. The syntax isn't what we're used to, but it works pretty well.
I know that SQL can do more when you separate its conceptual foundation from relational databases. It's an expression language, an interface language to structured data.
I used to work on a platform that could translate SQL into NoSQL databases and SQL databases alike, it involves translating the expression into something that fits the native data model. For something similar look at Apache Drill, Dremio, Hive or PrestoDb and how they hide different types of databases behind interfaces that make them compatible with SQL.
I can admit, shoehorning SQL as it is onto a non-relational source (ex. Graph databases or search) isn't totally clean. But I believe it can evolve further as a standard to address that.
It requires more work in middleware to interface between SQL expressions and heterogeneous technology. And getting creative with the language constructs themselves. The power of data projection, data shaping and filtering alone, not to mention aggregation, etc. is undeniable in my opinion.
Disclaimer: I used to work for a company that built data federation software which supported SQL in front of heterogeneous sources. So what I am saying I know to be factual in most cases (although difficult!).
I mean, I understand there's a use case where it can be useful, such as SQL query builders or APIs that support only SQL, but in general case when you don't need one interface to rule them all, what's the use of shoehorning everything into SQL?
Developers are also unable to become involved or voice their concerns in any meaningful way without spending an even larger amount of money to participate in their committees. Oh, and AFAIK all development happens behind closed doors, so good luck figuring out why certain design decisions were made.
> const selectOneBy = (table, column, value) => { const res = selectBy(table, column, value); return res && res.shift() }
Two or three lines gives you about 80% of the value in any ORM and avoids the -20% value found in the other 100k LOC.
I have a hard time believing this when the two lines above are obviously wrong. You can't use :column to escape the name of a column in SQL. So your code has to protect it, which is not easy if you wan't to be portable. MySQL will generally use `column` but may be configured to use the standard "column".
Anyway, selectAllBy() and selectOneBy() are not enough to be confortable. I appreciate the static completion in Something.find({id: 1}).complete, the relative queries like Post.find({id: 1}).author, and many other things that help against typos, help code faster and make the code more readable.
I agree with the OP that fallthroughs are much needed, because writing custom SQL is sometimes simpler, and sometimes much more performant. Usually, its mostly about writing SomeModel.findAllBySql() which most ActiveRecord implementations provide.
Sure. Though I'd love if ORMs would seal autogenerated classes to prevent inheriting from them, language allowing, or otherwise prevent people from using generated ActiveRecord objects as their business model objects. ORMs should be just for creating an OO view into relational storage. They should not be running business logic. I know that people worry about having to write another set of classes just to wrap the ORM ones, but really, sooner or later (usually sooner) your business logic abstractions will stop mapping 1:1 to storage needs, and then you'll be wishing you had that extra layer of separation.
Or, in other words, treat ORMs as a more verbose and somewhat-compile-time-checked SQL. Nothing more.
You don't have too. SQL is brilliant for writing queries that generate repetitive, standard SQL queries.
Provided that the database vendor documents the system tables.
person.orders.first
And you change your database to make an intermediary table when business needs change and you need multiple people to be associated with an order you don't need to change the above line of code, you just call a class method in person and everything continues to work. It only gets more powerful with scopes and other things. I still drop into SQL, but usually when I'm doing data analysis, not normal business logic.Edit: To clarify my position, Domain specific languages mean domain specific ideas. As soon as a product is no longer meant for general queries, but for queries over specific ki day of information in a specific kind of system, we can use that knowledge to make a powerful language in that domain.
The article was about ORMs, not Splunk, and consequently their DSLs
I've seen people crack out their hip looking graph-like DSL that they lay over the top of an Oracle database. It's crazy to do this. I know all this regular SQL so just let me use that as needed.
On the topic of query DSL’s - the only acceptable ones are those that are native to your programming language. A good example is the F# SQL type provider. If that’s sufficient for your needs, it’s actually quite elegant as far as query languages go - and it’s all F# meaning it’s not a specific language for querying.
Developers don't need other developers to tell them when or when not to use ORM. We know how it works. Some use cases are better for a quick inline SQL whilst others where you have hundreds of tables would lend well to ORM. It's not one size fits all and I'm sure we've all written a few apps in our time to know this to be the case.
And this idea you can just learn "one SQL" is pretty laughable. There is a subset of SQL that works across many (but not all) relational databases. But each database has its extensions which you really should be using to get maximum performance. ORMs automatically take advantage of these just to mention. And the behaviours you will get can differ immensely.
But if you use vendor specific extensions you will have “vendor lock-in”. What if someday on a whim your CTO decides to change the database your entire multi billion dollar organization from depending on Microsoft/Oracle and wants to go open source?
Yes I’m being sarcastic. In reality, statistically no company makes those types of sweeping changes because the risks are too high and the rewards are too low.
To me the saying "ORM is a Vietnam of computer science" rings true too often. Somehow projects which I deal with don't really benefit from ORM... so I could live completely with SQL alone.
I know about a software package that was written in C# that officially supported Sql Server,Oracle, and Mongo and he was able to support all three with basically the same codebase using Linq and runtime translated Expression<Func<T>> statements.
His LINQ queries and expressions were the same across data stores.
And would have been just as correct.
Of course today people understand this better and distrust ORMs increasingly more, whereas back 10-15 years ago then it was the heyday of enterprise Java and all the ORM craziness.
At least we're also over the NoSQL fad too...
Why is it that open source software which adds unnecessary complexity and slows down development tends to become popular but open source software which takes away complexity and speeds up development tends to not get any attention. It's like there is some kind of conspiracy.
all because of an inability to manage versioning change across the standard browser js api...
great for a giant megacorp looking to waste money on busywork and divide the workload across more employees (front-end devops?!?)
less useful when you already have to compile a backend, don't really have a problem with ES5, and understand that you are basically only tasked with making your dom updates dependent upon data model changes, YMMV
what does versioning has to do with bubbling, wtf?
In fact, the parent does see (and called) that "obvious problem with being stuck on a single version of anything forever".
That's why he doesn't cheer for the band-aid solution that's webpack and doesn't thing it "represents progress".
public List<Customer> Find(Expression<Func<Customer,T>> expression) {...}
and then your client can use any random expression...
Find(c => c.age > 18 && c.age < 65 && c.gender == “M”)
and have that expression translated to either SQL, MongoQuery, or anything else that has a LINQ provider.
If done correctly, not only can you switch between RDMSs, you can switch between an RDMS and a Nosql data store without changing any client code.
When you are writing your unit tests, you can use in memory Lists to mock your data store and still test all of your queries without using a database.
Of course you also get type safety, autocomplete, and in the case of NoSql data stores, you enforce a schema over a schemaless data store and get to work with strongly typed objects.
I’ve never seen any purpose of an ORM outside of LINQ. They just add another level of complexity and still don’t integrate with the host language naturally.
I should be clear though, from an architecture standpoint I think it's much better to go straight to SQL, and keep the data unnested. Like anything in life, it's a trade-off. It requires more up-front data model knowledge and more up-front work, but it's much easier to debug, more flexible, and performs better.
For juniors, it's best to have a less abstractions.
But I’m right on board with the “stop inventing rubbish pseudo query languages” idea. I’ve rarely seen an example that wouldn’t be better if it were it just “SQL subset with some additional features”.
On the other hand, there is a significant cost in terms of duplication. In ORM code it is trivial to decide a where clause at run time. Not so much in plain SQL. So you end up with a lot of copies of the same query, or else some sort of query assembly layer, at which point you're halfway to an ORM anyway. I am undecided on the issue.
The former category includes things like Django's ORM: very limited in what you can do, it doesn't actually "get" SQL, mismatch between the ORM API and the database. The latter category includes fewer software; sqlalchemy is my familiar example. Because sqlalchemy does not abstract the relational model, instead mapping it into your application language, it has ~few mismatches between its API and the database.
The former category is what "kinda works well enough for CRUD apps and I don't need to know anything about nothing" and then become a major problem, the latter category presumes you know how SQL works but then allows you to work.
Random snippet just because I feel like it.
def all_children(self):
cte = session.query(Tag).filter(Tag.parentid == self.tagid).cte(recursive=True)
cte = cte.union_all(session.query(Tag).filter(Tag.parentid == cte.c.tagid))
return session.query(Tag).select_entity_from(cte).all()And I agree. I wasn't involved in writing NRQL but statistics queries at New Relic during my time were already approaching turing completeness, so it was just a matter of time. Timeseries data in particular really doesn't map well to SQL at all, sadly, as a SQL fan working with a lot of timeseries data over the decade+ now.
I don't get this though. If you want to export your data as some flat file, why use MixPanel at all? Just store the events in your own database and query it yourself. Why pay the cost of both solutions?
Others don't because sometimes it's not that simple to fetch all of the data to give you with one click. Many of these companies are just rolling up the metrics in some online database and hiding the original files away in cold storage.
Of course some standardisation amongst ORMs would be good. That would also help them be less "garbage".
Any argument whatsoever for this? Would love to see an example of how when you do away with the whole object persistence layer, and you just have a SQL string (which ORMs allow you to do anyway), how do you get your parameters over, how do you get your rows back, how do you manage your transactional scope, how do you get your rows in and out of your objects? all of which has nothing to do with a SQL string.
Well let's read on:
"Erik Bernhardsson... is the CTO at Better, which is a startup changing how mortgages are done. I write a lot of code, some of which ends up being open sourced, such as Luigi and Annoy. "
Open source stuff! Let's go there and see an example of this "write raw SQL and don't use any persistence libraries and your code is readable and straightforward", because nobody ever seems to actually want to illustrate this and how they don't end up writing their own ORM anyway, and we see, oh https://github.com/spotify/luigi/blob/master/luigi/db_task_h..., it's SQLAlchemy ORM.
Foiled again in my search to see this elusive super clean and simple raw SQL with no ORM that doesn't reinvent an ORM anyway. Which is the real "myth" ?
https://github.com/HubSpot/Rosetta
Its by far the best Object <--> Sql layer I've ever used. You manage your objects in Java like you normally would and then Rosetta maps your objects to columns in your SQL DB using Jackson. This means you can write plain ole SQL for very clean simple code. When it comes time to get data out of the DB it again uses Jackson to map the results back to objects in Java.
I was super skeptical at first, but it makes reading/writing data out of a SQL database super simple, while encouraging very simple explicit sql.
You write regular SQL, can easily parametrize everything, and get collections of objects back. And it's very fast.
This is just a very low quality rant that appeals to base cheerleading instincts from whoever happens to agree with this person already, and I will choose the vast community of working software over this post in order to determine what exactly is a "myth".
I personally didn't already agree with what he wrote, but there was enough in it to make me think about the topic a little more and shift more toward his position. I already know what ORM code and raw SQL calls look like without him providing examples.
I do too and code that not only uses raw SQL (not that big a deal in and of itself) but also reinvents the whole persistence layer (which is the whole part of "ORM" that has nothing to do with writing SELECT statements or using DSLs) is a complete mess.
I really want to see the examples so I can learn from them and perhaps have it contribute back to my own ORM project (SQLAlchemy).
Delphi has always had a default non-ORM dataset layer (there are ORMs for it), and it works like this for queries:
1) There's a base TDataSet class (Object Pascal uses "T" to denote "Type") that handles all abstract column and row storage and manipulations, with no pre-disposed notion of how to actually load such rows. Dataset columns are abstracted away also, and provide a lot of automatic type conversions and functionality like BLOB column stream access. Having this base class allows disparate descendant classes to easily interact with each other without knowing the details of how the data got there. All query objects descend from this base class, and fill in all of the functionality for actually interacting with the database server/engine.
2) For query objects, the SQL string gets assigned to a query object property (typically called "SQL"). Parameters are specified as named parameters by using colon notation (:Parameter) instead of "?". During this string assignment, a property setter method parses the SQL string and automatically sets up the parameters as a collection that is another property of the query object. These parameters do not have any type information assigned to them at this point. The developer can choose to just manually assign values to the parameters using type-specific parameter properties at this point. Any type differences are managed by the underlying object, so assigning an integer to a string parameter will result in an automatic conversion.
3) If you manually prepare the query object using a Prepare method, then the SQL is sent over to the database server to be prepared and any type information about the parameters is sent back from the database server to be assigned to the parameters collection (depends upon the database server). If you don't manually prepare the query, then it is automatically prepared during the query execution (4).
4) The SQL is executed using an Execute or Open method. If the SQL is a SELECT statement, the query object is automatically populated with the rows and the rows are managed using the base TDataSet functionality. There are methods for navigation, updating, etc. and updates of non-directly-updateable result set cursors are managed by the query object. Query objects are linked to database objects, and the database objects manage the transactional scope for updates. You can also use such database objects to execute one-off SQL statements where the only thing you're interested in is the result set row count or the affected row count.
The interesting part is how Delphi handles static-typing of result set columns in your application. In the IDE, you can right-click on any query object and have it automatically populate "persistent" column objects whose class type reflects the underlying result set column type. This means that if you try to access or assign the column object's raw Value property, this access/assignment is subject to type enforcement by the compiler. However, you are free to use other properties that perform type conversion, in which case you'll get a runtime exception if the type conversion fails for any reason. You can also assign names to these column objects, allowing you to write easy-to-read code like this:
OrderCustomerID.Value := 1000;
So, you can get a lot of the benefits of an ORM using an architecture like this without having to sacrifice the flexibility of manually-constructed SQL statements. A lot of this depends upon the peculiarities of the Delphi component library and IDE, though, so how transferrable this to other environments is an open question.Not all DSLs are necessarily bad though. Personally I like to create some very basic and straightforward DSLs to replace horribly complex GUIs with tons of nested menus (aimed at professionals). I think DSLs can be useful for educated computer users who don't know SQL (like people in finance, medicine, statistics). Also there should be a limit to the complexity of a DSL. Not every DSL needs to be turing complete. Simple DSLs that cover the required functionality are the most successful in my experience with customers.
However replacing SQL just does not feel right, it's already an awesome language to query a relational database so why replace it? So, for the most part I agree with the author - although I miss the nuance a bit.
Don’t do much php nowadays but Django is a great db - object abstraction. You can go raw sql if you want quite easily.
I regret not adopting this practice sooner.
Most of my substantial work resides in that raw SQL feature. So, I use a library that offers good support of that.
I have written hundreds of SQL using query builders and ORMs.
Porting should not have been some onerous or career destroying activity. As Job would say I think you've made a huge mistake.
Most of my substantial work resides in that raw SQL feature. So, I use a library that offers good support of that.
I have written hundreds of SQL using query builders and ORMs.
Amazing how they got Bojack into that human costume.
My current theory to the proliferation of these query languages is ignorance. Aside from a severe case of NIH the developer behind our query language exhibited a flawed understanding of SQL. If you're planning to roll your own query language you probably shouldn't without first considering SQL, your intended users and the use cases that they're trying to satisfy.
Yes being forced to use a REST interface over reports is a bad idea but that has nothing to do with joins over a NoSQL data store.
Mongo does joins via the $lookup function and the C# Mongo driver will translate the LINQ code to MongoQuery for you.
https://www.axonize.com/blog/joining-collections-mongodb-usi...
Worse case even if you do get data sources from separate places, it’s just as easy to do joins on two lists as it is to do in Sql using LINQ.
SQL wasn't built for querying every possible data store.
They bastardised SQL to support JSON data types just like every other vendor has to do because SQL is for relational and only relational data stores. Is this the utopia that the author is after ?
You don't have to use the JSON/JSONB column types, they are optional.
We use them extensively in production and haven't had much difficulty learning them.
If you don’t want to learn - it’s a personal choice.
You can ride a bicycle that works everywhere while others prefer driving sport cars on a highways.
Additionally, you're making sweeping generalizations based on a limited perspective. Loading up the application layer is the right choice when your data will only ever be consumed by that application, and multiple versions need to coexist. When you're talking about data that cuts across a business, and where a single application version lives at a time, loading up the application layer is infeasible and unnecessary.
So now we have something like
CreateCustomer_1
CreateCustomer_2
CreateCustomer_3
And different apps are using different versions. If you have to make s change. You have to make a change to every version because you don’t know which app is dependent on which version.
What changed between the stored procedures and what code commit did they go with? Who changed it? (Usually this is solved very badly by having headers with comments that go back years). I can do a “git blame” on code, diffs, etc.
Loading up the application layer is the right choice when your data will only ever be consumed by that application,
Ideally, it should be one service per domain that is responsible for one set of data. That can be either a microservice or a module in a mono repo.
Of course reporting would be separate and would cross cut multiple domains and would usually have intimate knowledge of the tables anyway.
It’s also much easier/cheaper/faster to scale app servers than DB servers.
If you're going back and changing old versions of a stored procedure, you're doing it wrong. The only case where this would be true is a breaking schema change, and then application layer code would break too, so the point is moot.
Siloing your data makes it less useful (or rather, it increases the cost of using it) and less discoverable. Additionally, if you have some sort of security model in place on your data that becomes exponentially harder to manage in the silo'd case.
As for scaling - read scaling databases is REALLY easy. I admit that write scaling is more challenging, but stored procedures are read-dominant in general. For specialized or very high throughput cases, custom application layer logic can be made much computationally cheaper, but this requires more (and more expensive) engineers, so that must be factored into any potential savings.
Siloing and thick application layers are great for some use cases. Data aggregation and thick database layers are great for others. Insistence on one model in all cases is only going to limit you as an engineer.
And what happens when I need to rollback? I have to rollback the stored procedure code separately. What about regression testing?
If you're going back and changing old versions of a stored procedure, you're doing it wrong. The only case where this would be true is a breaking schema change, and then application layer code would break too, so the point is moot.
It would be one change in one module (in process) or one microservice (out of process).
Siloing your data makes it less useful (or rather, it increases the cost of using it) and less discoverable. Additionally, if you have some sort of security model in place on your data that becomes exponentially harder to manage in the silo'd case.
For reporting/analytics that would be a separate longer running process anyway and wouldn’t be a part of your OLTP system anyway and you probably wouldn’t be using stored procs. You would be using some type of reporting system.
As for scaling - read scaling databases is REALLY easy. I admit that write scaling is more challenging, but stored procedures are read-dominant in general. For specialized or very high throughput cases, custom application layer logic can be made much computationally cheaper, but this requires more (and more expensive) engineers, so that must be factored into any potential savings.
Yes, throwing up read replicas is just a matter of throwing money at the problem - and I’m not morally opposed to using money as an optimization technique when necessary. But you can still get more granular scalability with app servers.
For specialized or very high throughput cases, custom application layer logic can be made much computationally cheaper, but this requires more (and more expensive) engineers, so that must be factored into any potential savings.
And that’s where I come in :)
I've used a couple of ORMs and jOOQ blows them all away easily. It's not the first product that tried this approach but jOOQ does it really well. I've partly or fully converted at least 3 Hibernate projects to jOOQ and all my recent JVM projects use it exclusively. In my opinion products like jOOQ make ORMs redundant for most use cases.
BWAHAHAHAHA!!! No. I have to use ORMs at work because they are forced on me. Same thing with Eclipse, and GWT, and a million other things that I detest because my employer has structured their development processes around hiring average talent and getting average(ish), repeatable results (I'm serious, I work with a lot of developers who only know how to code in Java, in Eclipse -- it's maddening). I'm the malcontent who doesn't fit in because I say things like "Let's just use SQL." or "I promise you, this architecture is not scalable, regardless of your seniority over me."
I don't miss SQLAlchemy where I would trawl through documentation to figure out how to do X or Y.
As for SQL standardization, +1 to that. Learning a new pseudo-SQL is only fine if there are additional features on top of the DB, but I've encountered databases where the SQL support was present in the name only and extremely limited, with people making do with UDFs in order to achieve basic things.
Using sql to query a whole bunch of stuff has been very nice.
I just wish the excellent TablePlus client could use osquery under the hood to show a lot of system level things in a nice pretty UI.
Another reason to use a DAL like PyDAL is that, properly written, it can abstract away the data layer such that you can switch from (for example) MySQL to PostGres to Google Cloud SQL with essentially no retooling required.
Finally, using a DAL can help keep inexperienced programmers on track with security, accessibility, and performance requirements.
This is a sure way they'll never learn the real thing. I know since I'm sitting next to a (young but talented) guy who knows only Spring and JPA, and freaks out all the time because the developer of the legacy system had dared to assemble SQL queries manually in the code (which are complex enough I personally hadn't bothered using JPQL in a trial-and-error development model either).
Wasn't the post exactly about not wasting novice and other developer's time with crap criteria APIs and such? Just compare the sheer size of the SQL spec or reference manual against the shallow description of JPQL to judge which language you should be using for a project with any kind of depth.
One thing that really resonated with me from the article is the following:
> Every SaaS product should offer a plug-and-play thing so that I can copy all the data back into my own SQL-based database (in my case, Postgres/Redshift).
Totally agree, a data dump should really be a standard feature (and not take ages to produce either).
Sadly, being software, "the data layer sucks" is true of most data layers.
These developers find it easier to dip into shallow DSLs instead of learning the depths of SQL.
I'm sure lots of people do understand SQL. But if you started programming post 2010 it's entirely possible you've never had to deal with a row database. My case:
- GAE datastore
- MongoDB
- RethinkDB
Again not discounting those who've been programming longer, it's just that JSON-stores are becoming default and the amount of people who don't know SQL is probably more than you think.
“select * from customer where age > 18 and age < 65”
Any more of an abstraction and easier than
from context.Customer where c.Age > 18 && c.Age < 65
With one it’s a magic string where you could have typos. With the other you get compile time syntax checking.
If you had an in memory list you would write:
var seniors = from c in customers where c.Age > 65
If you were using an RDMS you would write:
var customers = context.Customers
var seniors = from c in customers where c.Age > 65
If you were using Mongo you would write:
var customers = database.GetCollection<Customers>(“Customers”)
var seniors = from c in customers where c.Age > 65
In each case, the query would be translated appropriately - as in memory code, SQL, or MongoQuery respectively.
If you encounter datalog in the wild it will be something pretty-custom, like datomic, pydatalog, or even differential dataflow (that feels more like map-reduce with fix-points?)
Hey. What do you mean? Which Prolog conventions?
(I write a lot of Prolog and I'm always curious what other people think about it).
I'll be using Gorm on my next project, because life is so short..
(I believe that it's (relatively) recently become possible to define tables as SQL data sources, not sure how practical that is or if it's used at all.)
Excel can query external data sources with SQL queries, and pull the results back into the spreadsheet.
And via ODBC you can use .xls files as data sources, and query them with SQL style syntax from other programs. (I don't know how far beyond the basic "SELECT [Column Name One], [Column Name Two] FROM [Sheet One$]" it can support).
Both of those date back - https://www.connectionstrings.com/microsoft-excel-odbc-drive... - most look like Excel 2000 or Excel 97.