I think that making semiconductors is unfortunately out of the reach of hobbyists. You'd have to grow purified crystals (or buy them), slice and polish them, etc. The various processes needed for lithography involve toxic chemicals too AFAIK. That being said, if you want to make your own large-scale CPU, you can still build one out of discrete 74-series logic, individual transistors, or relays. Takes a lot of dedication, but that is entirely within your reach.
A fun example of a CPU built with discrete SMD transistors, the monster 6502
https://www.youtube.com/watch?v=tQIwS2GzXLI
There's also the MegaProcessor: https://www.youtube.com/watch?v=lNa9bQRPMB8
Zusie relay computer: https://www.youtube.com/watch?v=NXeBR-lbnjI