So this has actually always been the case. Hardware firmware blobs that initialize the device to talk on the bus yielded to EEPROM chips that had SW firmware burned into it to do that initialization as the EEPROM chips became cheaper and let the bringup process be more flexible. This yielded to microcontrollers that had programmable flash (still some EEPROM for some parts of HW, but now you can glue multiple pieces together). Now you have full-blown ARM cores doing that orchestration (but HW still has EEPROM for bug fixes on top of the base NOR-flashed code). It’s all about flexibility and cost.
I don’t know though if I fully agree with the characterization that Linux has been shuttled off to the corner as chip vendors work around it. It’s more that the computer has actually always been a distributed system and OS/hardware developers have ignored that for a very long time. You wouldn’t say that Linux isn’t the OS for distributed clusters even though a lot of work goes into making those clusters run and that all typically runs in user space outside the OS boundary.
It’s possible that there’s a better model out there that manages to unify things but I’m skeptical. All this code runs on different chips with different clock speeds. Code also resides in drastically different memory spaces and that’s unavoidable - an M3 might be able to load the firmware over PCIE, but it’s not executing out of main memory. Additionally these chips typically have drastically different cache coherency properties.
Now maybe we need to revisit this all holistically, which I think is the actual pitch being made and one I can support. That needs to look like defining interfaces for how this stuff works (& yes, making it possible to integrate into Linux) and getting chip vendors to adopt. Sunk costs are real and trying to rework not just HW architecture but also the entire SW ecosystem can be a dead end. I’d certainly be keenly interested to hear the speaker’s ideas. He’s certainly far more knowledgeable about this space than I am.