Hacker News Reader

In one of my previous companies, they used Airflow(by airbnb) to schedule and manage workflows. Previously they were using nothing but CRON. It turned out to be not so efficient to retry failed workflows and cancel the execution of following dependent jobs. Airflow turned out to be a great fit for our case. I highly recommend checking it out.

View on HN

We're using Apache Airflow [2] internally as well. It's pretty featureful and addresses most concerns mentioned by OP, such as orchestration, retries, open source code reuse, and dependency management. It has primitive monitoring and alerting, but one needs to bring something external for that today.

Shameless plug - My startup [2] offers Airflow as a SaaS as well as an enterprise distribution with monitoring tools to build upon core Airflow.

[1]: https://github.com/apache/incubator-airflow

[2]: https://www.astronomer.io/

Reply on news.ycombinator.com