You probably only ever hear of the success stories. I can think of Terry Tao, John von Neumann, and Erik Demaine off the top of my head, not because they were prodigies, but because they have gained enough name recognition that I Google'd them and learned of their early education.

Even if the rest had very interesting research, you wouldn't hear about them outside of their chosen field unless they reached superstar status.