30 years ago, if you worked in software, then you were almost certainly building shrink-wrapped products, that had to "go gold" before being printed onto 1000s of CDs at a factory. How quickly/easily could you change anything? Not for years, typically. Hence, you were very careful to try and build things right the first time, but the ironic thing is, with such long feedback cycles (i.e. you never really got feedback), that all this waterfall-style effort to plan everything and do everything right the first, rarely ever materialized as perfect/seamless UX for the end user anyways. It turns out, you need to iterate to improve.
Nowadays, the pendulum may have swung too far in the opposite direction... it could legitimately be said that, some things are too easy, tinkering with changes is too easy, and thus nobody expects to have to maintain something for a long period of time. If the cost of rewriting everything is low enough, people will be tempted to just do that whenever they change their minds about something, rather than forcing themselves to predict the future upfront and make durable decisions that will survive for 5-10 years out in the wild with no changes.
IMO, there's no single right answer for all scenarios, all products, all forms of engineering, etc. But it's explainable why such trends may emerge over time.