Smart-ass answer, but the fences are financial and very real. If you have no money, how are you supposed to afford to travel to a completely different part of the country where you have no friends or family, find a job and rent an apartment?
PS: The first US colonies actually took a crazy amount of capital to set up. The west was largely settled by people in covered wagons which for the time period also represented a lot of capital ~50,000+$ in 2018 USD.
But the west was first colonized by the Spanish.
One of the first Anglos to reach Alta California was the Bartleson-Bitwell party https://en.m.wikipedia.org/wiki/Josiah_Belden
For example, basically all of New England that isn't on the coast was settled by people who got so fed up with the puritanical authoritarians in Boston that upending their lives and moving a few days ride west (and risking starvation/financial run in the first few years) was the better option.
The underground railroad gets a lot of coverage, but it's peak never reached 1,000 slaves per year where 23,191,876 slaves where part of the 1850 sensuous. So, people are talking about a migration of ~0.004% per year.
Cheaper land makes being a farmer vastly easier. The same capital allows for more either more farm land or better farm land. Remember, farms are just like any business you can extract more money when the output stays the same but your costs drop.
Also, saying significant wealth is required to setup a colony is very different from individuals having that wealth. Consider the wealth required to build a 80 story high rise generally vastly outstrips the wealth of people living in it.
PS: That said, looking at modern immigration even illigal immigrants to the US tend to be better educated than the average peopulaion of their origin countries. Likely becase immigration is a lot of effort, weeding out the less motivated and capable.
Twenty-two dollars.
That adds up quickly.