Mostly, I was educated in the Québec system, and had no idea before I was 16 that if I got good grades I could get into interesting universities -- even the American ones. Reading up voraciously on Richard Feynman, only to discover that if things had been different, I, too, could have gone to that school. By that time, it was too late, I simply didn't have the grades. My parents' perception that it would cost a fortune certainly didn't help.
Not that I'm complaining, I can't change the past. But the idea of never attending MIT is one of those very few, recurring what-ifs? that will sometime keep me up at night, half a lifetime after.