I think the replacement for the high school system should be based on similar principles as the chess clubs. I don't know if there is such a thing in the US so let me tell you a few words about how chess clubs worked in Poland (and I assume in all countries of the soviet block).
First of all, they were open to any child. I was never any good at chess, but I had this friend who was quite good so I wanted to be like him so I decided to enroll. I showed up and declared that I wanted to become a chess player. They told me to play a game against my friend to asses my level (I remember at the same time there was another kid who came without a friend, so they made him play against a computer; the chess computer was big, easy to beat and manufactured in East Germany). Then they assigned me to a group. The number of children in my group was varying between three and five during my time there. Each group had a coach. Our coach was an international master and former champion of Poland--it's a bit like assigning university professors to teach you fractions; but that's how it was. We met twice a week and our coach would teach us all the important stuff, openings, endgames, combinations, strategy. He would analyze our games with us and give us homework. He would tell us what to concentrate on and gave specific tips on the effective practicing. The club had a nice library of chess books. It also organized tournaments. And, the clubs were competing against one another.
It was fun. Not all kids were good players, but we were passionate about it, we would play blitz tournaments, discuss various topics and argue about our favorite openings (which felt a lot like arguing about programming languages, now that I think about it). The fact that we got to meet other kids who played chess made us more excited about it. The whole experience was wonderful on many levels and, while not all of us made much progress, some did.
I imagine how great it would be if instead of compulsory schooling there was a system of math clubs, hacking clubs, physics clubs, chemistry clubs and so on. I don't think it would cost more than the current system.
Now, you may object that since this system would not involve coercion, many children wouldn't enroll to any clubs and they would remain terribly uneducated. I don't think in reality that would cause much difference. Let's do a thought experiment. Imagine you approach random 100 people in your city and ask them to complete two tests: one on chemistry and one on chess. Both tests would cover very basic questions as well as more advanced (of course, the decisions which questions represent the same level of difficulty is arbitrary, but let's skip over it and say we are using some common sense approach). What do you think the results would be? Would more people know the answer to "How does a Knight move?" then to "What is the valency of carbon?"? How about, say, King's Indian Defense vs. nucleophilic substitution? Personally, I think that on average people would score better on the chess test. But I have never really performed such experiment, so I may be very wrong here.