While my experience is mainly in digital hardware, my tinfoil hat theory as to why the hardware startup scene is so lackluster is because the EDA tools are expensive hot garbage. Not only are the dev tools cost prohibitive, but so are the necessary analog components like PLLs and PHYs (which are more or less required in pretty much every modern VLSI design.)

The same companies that make the EDA tools also make IP blocks that semiconductor companies purchase, so it's in their best interest to prevent an open source hardware scene from thriving. I get that making masks for a chip is expensive for a startup. But if the design, verification, and layout steps were cheaper and more streamlined, I would bet money that the fab costs would work their way down in clever and creative ways.