I would go futher and say that Agile exists for the particular constraints of UI software development. User stories, Product Owners and Agility break down if you're building core infrastructure
> Engineering systems requires an engineering process and discipline, and agile is just not up to the rigor.
Agile just means closing as many feedback feedback loops as you can as early as you can, and to preserve opportunities to course-correct for as long as you can. All this to keep the destructive effects of surprises (which are sure to catch you) as low as possible.
This is good engineering practice, plain and simple.
1 - Is it a domain where "user story" means something? 2 - Is it a domain where you can iterate fast?
Usually for UI these two conditions are met. From my experience, when talking about big infrastructure projects, they aren't
All else equal, smaller or shorter feedback loops are better. And often shortening a feedback loop is worth considerable tradeoffs elsewhere.
It's certainly the case that being able to iterate fast may not be currently possible, but being able to do so, or even just being able to iterate faster should probably always be a priority (even if not the top one).
This all applies even to physical real-world infrastructure. I remember a story about a city somewhere (South America?) where the mayor (?) was able to shorten the feedback loop whereby roads were built or repaired. That seems like a good, even great, thing, even at somewhat considerable cost.
What kind of big infrastructure projects were you thinking of specifically?
Maybe a bad example, but it's basically mandatory to include a manpage for a cli utility, but do you always include drag & drop for a file uploader?
We have an API that is used by external systems, those systems (and developers) are our "users".
It's vital to think of API consumers as users, because it avoids the "REST is CRUD" attitude by focussing on the business/system functions that are being expressed, which leads to focussing on the Nouns, not the verbs of the business.