As a society, we made the decision that it is not fair for children of richer parents to be able to have much better education that the rest, and instead made it better for everyone.
Since you insist on stating it in terms of freedom, we traded freedom from the obligation of public schooling for the freedom to have high quality education (and thus equal opportunity). Thanks to this choice, we have better outcomes than the US, and just as much freedom - just not the reductive kind of freedom you recognize.
If you really want even better math education, the solution is that instead of shelling out money, you can do research on how to ameliorate teacher education or pedagogical techniques, or run as a politician to get those resources allocated. That is also your freedom.
Ultimately, if you care about creating something resembling equality of opportunity and if you want to reduce inequality, you will have to abandon your reductive concept of freedom, where a homeless man starving enjoys more freedom than if he was in public housing, somehow.