I am also white, and I also grew up poor. I agree that many (though far from all) of the college students who talk about socioeconomic inequality are spoiled brats.
However, you have misunderstood what "white privilege" means: it's not that your life has been easy, it's that the same situations would have been even harder if you had not been white.