Dagster vs. Airflow

I'm not sure we can label one solution to be "better than" the other just because it "is leveraging containerization to entirely solve..." anything. Reasons:

1. Containerization addresses the monolith pattern issues yes, but introduces other challenges (data persistency, image management, immutable infra, etc) 2. It takes much more than just containers and executors to design a reliable platform for ML/Data.

I think what can set projects apart is how/if they make use of Software Engineering patterns for what matters to ML as a workload. An orchestrator that could leverage a control loop (reconciler) pattern, similar to what K8s does for containers, would provide the eventual consistency and reliability required by most ML workloads.