Why do you think that is the case? What sort of benefits do you feel you are you missing out on?
1) The professors. Imagine taking classes and interacting with Nobel prize winners, or other famous people in your field (let's say, taking a class with Knuth at Stanford).
2) The networking/social aspects. Imagine being college roommates with "the next Mark Zuckerberg", whoever that may be.
3) More cynically, the pedigree. This one isn't strictly required, but it helps enormously if you want to get into academia, and it can also help you with an industry career or start a startup (it can make things easier).