These are the things engineers disagree on: the structure of these, how much of them to do, what constitutes "good" or "necessary" practice (I have a friend who is fond of saying that every place he's worked that claims to use scrum engages in "scrum-but" in practice. "We do scrum, buuuut the daily standup is a waste of time." "We do scrum, buuut we don't do sprint planning; we just have people pick up tasks on a common queue as they come up dry." And so on.

(Indeed, I've worked in industry sub-sectors where some of these best practices are widely understood to be counterproductive. Videogames often do few if any unit tests, preferring instead to rely on an army of human testers reviewing every build over and over because you can't generally capture the je ne sais quoi that is the feeling of "fun" in a test or even a design doc. By the time they've rapid-developed until they have a prototype that feels right, that prototype is the product and it doesn't make sense to add tests at that point because you can just test that one artifact to death by hand and ship it. This does, indeed, damage the reliability of the end product and result in something that often needs a week or two of follow-up patches... But because for all they complain, gamers don't actually stop buying games day 1 because of the bugs, the industry as learned leaving them in is acceptable risk that saves time).

Well, that's the thing. Where the money is is in novelty, not maintenance.

If we shifted resources to supporting maintenance over novelty, this could change... But the people with the money want to make more money, and that mostly happens in the "come out with something novel and everyone gives you a buck" space, not the "keep the electrical grid control architecture running" space.

Incentives drive the industry.