You already can get your own designs fabbed if you really want and can afford it (several k$):

https://en.wikipedia.org/wiki/Multi-project_wafer_service

There are some great services out there for people who want their designs made at that level. Speaking as someone who has worked in fabs - there are so many ways it can go wrong and so many ways you could kill/poison/burn/electrocute yourself if you tried to do it yourself. Your lithography would probably need a lot of tweaking too!

All it would take is for lithography equipment/chmicals/etc to start being sold to anyone at an affordable price... so, maybe in 50 years.

On that note, you could technically just buy the chips and spend an inordinate amount of time and money building the PCB and firmware.