They also want a rather remarkable amount of (expensive) paperwork sent to them. Everything about the SAT costs money, $50 for the basic test, $11 to send the scores to a school (seriously, $11? In 2013 it costs $11 to shove some digits somewhere?) although all is peppered with "fee waiver available".
Aside from the cost investment, there is also the time investment problem. As per above if you decide you can't afford it, and they want a fairly large amount of work done to apply, even if even more paperwork could result in it being free, well, why bother?
My brother went to Harvard, partly because their financial aid offer was best, and I recall my father cursing the required forms. He's a lawyer, so ordinary legalese doesn't phase him, but he found those troublesome. Now I imagine a family without a JD in it, or even without a BA in it, trying to fill them out...