I'm increasingly of the opinion that "basically a data warehouse, but incrementally maintained materialized views under the hood as an implementation detail" (e.g. Materialize, Feldera, anything based on Differential Dataflow / DBSP) is a good point in the design space. I get that they're more complicated than batch, and for many orgs the low latency isn't necessary for pure analytics/dashboards.

I wonder if there's a place for such architecture in the operational path, not just analytics. If you squint, "this microservice reads data from this operational DB and passes it to this other microservice / puts it in a cache / sends it in response to a request" looks sort of like an "ad hoc, informally specified, bug-ridden, slow"[0] implementation of incremental view maintenance (in that a cache, or a service's data model in memory, is a sort of "materialized view" over the source data). I've seen some success in replacing a tangle of imperative-languages-and-network-requests with a declarative, incrementally maintained model in SQL, if the read latency can be kept under control.

I'm curious about the "some amount of streaming influence moving back into the batch systems, particularly as object storage gives us the ability to run stateful workloads with less operational headache" part. could you expand on that?

[and yes, I do find these sort of problems interesting! :)]

[0] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule