Why don't policies go through double blind trials or more A/B tests? And attach automatic rollbacks to policies based on metrics. I'm all for trying new solutions but they should be iterated on quickly.
Less facetiously: A/B tests are great at lots of things. But these results really shouldn't surprise anyone. The idea that you will improve things for one group of students by making things worse for another group is really not only silly, it cost real kids a year of potential progress. An A/B test would cost some percentage of kids that year, which is better than hurting the whole group, but A/B testing on something that obviously would hurt their future does not seem fair to those hurt by the proposed changes.
Even ignoring the consent and ethical issues, doing something double-blind in a government ran system impacting statistically significant number of children simply can't be done. You can't maintain the blinding.
'Automatic rollbacks' are, frankly, absurd. How do you 'roll back' a child in 8th grade that failed algebra? Your quick iteration turns into dozens of cohorts to be tracked and managed.
A question to ask is it better for a kid to score 90% in algebra and it takes 2 more years or is it better that they score 50% and then move on to calculus. That would be a good A/B test to compare their lifetime earnings. My hypothesis is that calculus is a pretty useless skill and basic maths is more important in life.
There could be ways of doing A/B tests if the standardized tests are the same between states that have different curriculums.