We did it that way until the early to mid twentieth century, when a populist backlash against that good ol' boys system led to more or less the primary setup we have now. It still varies a lot year to year and party to party.For instance, the Republicans' system was quite a bit more democratic than the Democrats' this year. (You can draw your own conclusions from that on the relative value of the two models.)
It's an odd hybrid where the parties are private organizations that can make their own decisions and rules, but their leadership elections are run by state apparatus. This (understandably) confuses people, to the point that most people seem to think of the parties as almost part of the governmental system.
As others have noted, this all goes back to our winner-take-all elections, which are a kind of original sin in our system, which we'll probably never be able to change. In other systems, which encourage more parties, it is much easier to abandon a party if you don't like the leaders they pick, so it is not as important to be invested in the actual process of picking those leaders.
The Atlantic had a really good article on some of these topics recently: http://www.theatlantic.com/magazine/archive/2016/07/how-amer...