Part of the problem is that the context for the questions involves things that affluent students are familiar with. As I understand it, that can have a much larger impact than what one might think:

For example, imagine if you (or many HN readers) took a test with questions framed around coding ('I'm going to own this', I'd be thinking), or the same questions in an abstract sense but framed around the details of working in an oil refinery. Now add that you are 16 or 17 years old, from a family where nobody has taken SATs or gone to college before, wondering whether you belong, and under enormous pressure to do well enough to get a scholarship. Your future - whether you live as a college-educated professional or as a career Amazon warehouse picker and Uber driver - depends on it.

A major reason kids like that drop out of college at high rates is that they feel isolated and like they are living in a foreign culture with no other 'ex-pats' around. College is a big step for affluent kids but everyone they know has gone, and the campus and town are built around their culture; few or no college environments are like the poor side of town. The non-affluent kids feel like they don't belong, even that they are imposters (and when they go home, their new experiences make them fit in less and less, and people there question why they are wasting time and money). Now imagine those same kids taking the SAT.