You need to break up. With SQL

Counterpoint.

I worked tightly with the "Head Data Science" person quite a bit. Without fail, they eould end up seeing anomalous as all get out results on a regular basis. On further examination, it turned out that the devs were using an ORM that "protected everyone from SQL".

Yet every last time there was a gnarly issue to debug, the answer inevitably was "turn on logging of the emitted SQL". I could plop the emitted query in front of them and it would take no longer than 2 minutes for her or I to go "Ah ha!", then off they'd go down the road to fixing their stuff.

The relational model is incredibly powerful, and while I fully acknowledge that "code in the DB" as PL/SQL (stored procedures) is fraught, whereas the tooling that allows one to combine things like Python with the relational model has created an uptick in version control of data scripts alongside code; I cannot take seriously the assertion that SQL/the relational model is something to be abandoned.

At some point, you're still dealing with relational algebra. You aren't getting away from that. Even PySpark under the hood is still utilizing SQL. So what would you propose replacing it with?

I'll fully admit the lack of enough imagination to imagine what it would look like.

You still have to define a datastore endpoint/protocol to communicate with it.

You still have to describe the data being sought. You still have to support navigating the semantic space of your schema.

SQL allows one to use any of a myriad of reliable relational databases, coded by people who live, s*it, and breathe performant and reliable data storage, whilst defining a common query mechanism that allows the much less passionate about the arcane vagaries of databases to build up large, arbitrarily complex, yet appropriate to the task at hand data stores.

SQL is to reliable data storage systems, what Excel is to the financial/business world. It is the rock underlying pretty much everything we do. It is foundational.

Abandoning it though seems most unwise.

Especially since besides Excel, SQL actually has more infiltration into the general population than basically any other programming language ever concieved by man.