You did mention racism but I would also say racism was a driving force since before white flight. The government precursors to Fannie mae and Freddie mac wouldn't back mortgages that banks created in redlined areas. And the precursor to HUD created segregated public housing, including in the west. The Federal government was pushing redlining and segregation in housing since at least the Great Depression. A lot of this is covered in "The Color of Law" by Richard Rothstein.