Replace “Airflow” with “Linux,” “data engineers” with “systems programmers,” and “Astronomer” with your hypervisor of choice (Xen/VMWare/etc.), and you can see how absurd the author’s point is:
My problem is that ~Airflow~ Linux was not designed to address [high-level systems architecture] problems. We don’t need a better [Linux], but we need a higher-level one: a system that enables ~data engineers~ systems programmers to think at a platform level.
In fact, [Linux] is already displaced. [Linux] qua [Linux] is already obsolete, and it happened right within the [Linux] ecosystem. It’s called ~Astronomer~ Xen/VMWare/etc.
If it sounds like you could simply replace [Linux] with basically any other ~job execution engine~ operating system, that’s because you could.
This is where the argument falls apart. Yes, for very large, complex deployments, higher-level orchestration is important, but the choice of low-level execution engine is also still hugely relevant, just as the choice of guest OS is still hugely relevant when discussing large deployments of VMs.Furthermore, very few people actually need very large scale deployments; user experience and capabilities at the low-level are what most users actually care about.