At my firm, we build complex data pipelines (data goes from S3 -> Spark -> Redshift -> between Redshift tables etc.). Our data sources comes from FTPs, vendor databases, vendor flatfiles, etc. Further, business analysts build Tableau dashboards on materialized views or temp tables. Data scientists use raw data tables and create aggregated tables or forecasts.
It would be tremendously helpful to have a way to map out programmatically where the data is going and coming from (which vendor FTP did this table come from, where is this view coming from, etc.)
We have utility functions/libraries (use Scala & R) that we always use for reading data, writing tables, etc. It would be amazing to just plug in API calls into those utility functions to auto-generate maps which we could later edit/document. The new trends of data catalogues (see https://alation.com/ or http://data.world)is really useful - but it requires a lot of manual documentation (non-programmatic) that data scientists inevitably don't do.