Data Hubs.
It’s queriable like a database but it doesn’t store your data - it proxies all your other databases and data lakes and stuff, and let’s you join between them.
Trino is a great example.
It’s queriable like a database but it doesn’t store your data - it proxies all your other databases and data lakes and stuff, and let’s you join between them.
Trino is a great example.
Aren't stuff like data lakes and warehouses supposed to address the need for a centralized datastore?
Outside of perhaps an easy-to-apply interface, what benefit would a data hub provide over just streaming duplicates from all of your databases into a single data lake like Snowflake?
We used to have to copy and shuffle data into centralized systems so we can query and join it.
Data hubs do away with all that. Stop needing to think about storage. Instead, it’s just about access.
There have always been fringe tools, eg I once did some cool stuff with MySQL spider engine. But modern systems like Trino (formerly called Presto) make it easy. And, I predict, will hit the mainstream soon.
100% resonates when you put it that way. Thanks for the explanation!
(not affiliated in any way). https://trino.io/