This is inaccurate. The U.S. had a highly restrictionist immigration system from 1921-1965. The foreign born population dropped from almost 15% to under 5% by 1970.
During that time, the U.S. had a small number of highly skilled immigrants, such as German scientists fleeing the Nazi regime. You’re talking about a very small number of truly exceptional people. A $100k/year fee is not going to shut down this kind of immigration.