> "spicy" take: allowing users to write imperative code (e.g. using loops) that dynamically generates DAGs are never a good idea.

Said with all love: you're definitely wrong and it's not hard to demonstrate.

  1) some upstream service exports data as a series of chunked files each day, you can't know in advance how many
  2) processing a single file pegs a CPU core for ~an hour
  3) you want a task per chunk so you can process them in parallel
  4) ergo: you NEED dynamic task generation

This is a thing that Airflow famously couldn't do until 2.3 or 2.4. Luigi could always do it. Prefect, Nextflow, and Martian(?) can do it. DBT can't do it, though the concept doesn't exactly apply (the DBT version is more like "I want to generate models at compile time" which is much more debatably sane... I have this problem currently and our solution sucks, but mostly I wish I didn't have the problem!)

I see where you're coming from, I think, but my experience has been the opposite. In any major project, I absolutely need my DAGs to have a dynamic shape. Forcing them to have a static shape just pushes the required dynamism outside the tool, likely into some nasty code-generation step.