I would argue the opposite problem: those who are interested in math at the college level are forced to sit through two years of applications-based mathematics before ever touching the theory behind it. Nearly all of higher-level undergrad mathematics is hidden behind courses designed to teach engineers problem solving techniques using calculus. As a result, the intro calc courses don't do a particularly good job of imparting either theory or applications, unless you have an unusually talented professor. I came out of Calc II being able to solve some nasty integrals only to realize that you really only need a couple simple techniques, and with no idea how to set up problems until I went through a few physics courses. What was the point of all that - we could have learned integration by parts and u-substitution in a two week session and moved on, or actually spent a semester of analytical work preparing for higher level math.

(edit: differentiation -> integration)

My pet theory is that math is not taught well because teachers for the 6-12 group just aren't good at math. They may be good at pedagogy but may lack the deep understanding that may be needed to teach more effectively.

Now can society can divert a bigger proportion of the few people that somehow got it to teach kids?