> even VC funding for starting a business is tied to this
This is the main problem.
Excluding VC, businesses exist to make profit. The purpose of IT and engineering is to enable that business or make it operate more efficiently - engineering and technology should never be a goal in and of itself.
Given this situation, the most efficient technology (that enables faster delivery of business software) would win. "Shiny" stuff would never win in terms of mindshare unless it's actually good and enables faster delivery than the status quo.
Now add VCs to the mix. Suddenly you have an infinite supply of free money that distorts the usual market dynamics, allowing businesses to keep operating with no regard to profitability. The downwards pressure of the market pushing for businesses to operate more profitably is no longer there. As an engineer, you no longer have the pressure of building technology for the purposes of making the business more efficient - you lost that valuable feedback loop.
Now run this for 10 years. Entire careers will be built on this - why even try to build a profitable business when you can be a "serial startup CEO" and merely use the right amount of shiny to solicit endless VC funding to continue the adventure? Imagine a well-meaning, budding engineer joining such a startup and learning such a corrupted idea of "engineering" with none of the usual objectives, pressures and feedback loops of engineering.
In the end, you end up with "senior" engineers (with over-inflated titles) whose entire careers are built on technology that wouldn't actually pass muster in an actual business under the normal market pressures. But they don't even know anything else or why that is wrong (because they've started their career during this charade and never had any taste of what real engineering is), so they keep pushing for these (now-inadequate) solutions, deluded that they are doing the right thing.