> Consumers would just use the underlying foundational models directly to solve their problem

Whether this could happen is a critical question.

What ive found currently is that its not possible because the non-determinism of the model means you cant trust it, and these tasks are such that people expect determinism. Even if they dont then they dont tolerate the kind of stupid mistakes these models make

What could happen is that the model becomes good enough to be near deterministic, or as deterministic as humans. I dont know if the current next token prediction foundation is sufficient for that. But who knows, maybe it is