Models are stochastic. Did you look at pass@k? I wouldn’t be surprised if you saw a regression because these models are extremely complex and impact of various decision making downstream is complex.
Gemini's answer was very opinionated and factually correct, whereas Claude gave a more nuanced answer, which was also very good.