They moved to CA in the 1840s to seek to get rich, not because they were already rich.
People moved to CA to try to make it big in movies or music, although most failed, they still lived a pretty good lifestyle throughout most of the 20th century. Not because they were already Brad Pitt or Madonna.
People moved to Silicon Valley, initially just to get jobs at a burgeoning semiconductor industry a few decades ago. Then later to start up companies in personal computing or internet businesses. Not because they were already Steve Jobs or Jeff Bezos. Although most people never got that level of success, there was still a pretty good lifestyle available to young adults in the valley throughout most of the late 20th century.
Most people throughout California's history moved there for opportunity not as a luxury.