HNHacker News
TopNewBestAskShowJobs

msinxi

2 karma · joined October 8, 2021

submissionscomments
msinxi··on ETL Pipelines with Airflow: The Good, the Bad and the Ugly
Checkout great expectations (https://greatexpectations.io/). You basically run assertions on your data. It auto-generates documentation for you and you can also store results of your validations in a flat file or database. I think there are dbt modules that try to mimic great expectations' validation but I like the docs that come with great expectations