I don't think "answering math questions" is a good or interesting test of LLMs, especially prime number checks. Whether it ends up with the correct answer for any given question is largely down to luck. It simply cannot do basic arithmetic reliably, even with chain of thought. It's also, even if its performance at that task improves, a terrible waste of computational resources. Nobody would ever use it in a real world application -- at best you'd ask it to write a program that can determine it for you, something which it is reliably good at.