Or, if the "tabular data" is heavily relationship-based, then possibly replace "relational data warehouse" with "graph database", and "SQL queries" with whatever querying language that graph DB is natively / most expressively queried in.
Of course, this is the most important implicit "equally important" factor, one that an ML dev would think goes without mentioning: the generality or "power" of the model in what questions it can answer. You can only make these trade-offs in the context of knowing what kinds of questions you want your model to solve for! If all your questions are quantitative ones, maybe the right "model" for you is an RDBMS!
---
Though, that being said... why can't a deep-learning model emulate the thing that an RDBMS does, "at runtime", as part of its "mental toolkit" for approaching problems? That would be the best of both worlds, no?
I know that LLMs in particular have been observed to have "emergent numeracy" above a certain training-set size. There is a step function in how they approach such problems, going from their only being able to answer arithmetic questions on numbers of bounded size, and sometimes getting the answers wrong (probably this is due to a memorization-based approach); to being able to answer arbitrary arithmetic questions on operands of unbounded size, and always getting the answer correct.
I would guess that that what's happening, is that they are developing a functional component of their network that works akin to an Arithmetic Logic Unit, operating not on tokens, but on tokens transformed into a "numeric register" representation that is amenable to having math done to it with stable, quantized, position-independent results. (Just like the functional component that human brains develop after seeing enough math problems... probably.)
Do you, as an ML dev, think it would ever be possible for any of the model architectures we're familiar with today, to be trained such that they would develop an analogous emergent functional component for handling tabular-data questions, by transforming its internal working state into relational-DB/graph-DB data structures — e.g. page-heaps of binary-packed row-tuples; B-tree indices; etc — and then manipulating the working state in that form, using learned algorithms applicable to that type of data?
It seems to me (possibly just because I don't know any better) that just as with numeracy, "being able to put the data into a different and better internal representation" is what would be needed for deep-learning models to become truly good at dealing with tabular-data problems.
But, unlike with numeracy, "thinking as if you were a relational database" is not something a single human would ever intuit how to do without being taught. Relational algebra — and the data-structures and algorithms to make it practical to have a Turing machine do said relational algebra — wasn't even a single intuition, but a conscious effort, of multiple humans, working together over years. I strongly doubt that there's any number of "tabular-data problems" that you could show a human being, that would result in them developing an intuitional ability to do what a relational database does with its memory to efficiently answer queries.
(I suppose we could give an ML model an RDBMS, and hardwire it to interact with it. I know there are hybrid ML + formal-logic systems. Are there hybrid ML + data-warehouse systems? Not where the model queries an external DB — while that can be done, it'd be only in the same "stop and do this" way that ChatGPT runs Python code, which wouldn't make it a thinking tool the way that the formal-logic proof engines are for hybrid ML systems. Rather, I mean that some data-warehouse execution engine could be embedded into the ML execution framework itself, deployed as part of the GPU shader-program to each tensor core, such that data-warehouse operations can be done as a native part of the network's per-node instruction-set. Anyone ever tried this?)