This history of the Western Addition (including the Fillmore) — http://www.foundsf.org/index.php?title=Western_Addition:_A_B... — suggests that three factors contributed to the growth of its black community after WWII:

(1) a sudden rise in SF's black population in the aftermath of WWII (from ~5K residents in 1940 to 43.5K in 1950)

(2) the Western Addition's already diverse racial makeup (relative to other neighborhoods at the time)

(3) the internment of Japanese Americans, which removed ~5K residents from the Western Addition virtually overnight.