Anyone have a good data lineage story while still using mainstream data processing and ETL tools like Spark/Flink/etc.? I assume they may be selling something like that, but their "integration with my existing data stack" is pretty opaque.
To me, that's the core part (As the article also mentions) that allows data discovery. Otherwise you're just guessing with heuristics [0]. It also allows someone to go from map [Input Table(s) => This Table's Fields] which I think is half the battle in understanding a table.
[0] Not to say guessing is the worst, just that the information is already in the ETL code so why are we guessing?