Counterexample from only 2 months ago:
And it can count fine. It doesn't know how to spell.
Counting most certainly is math.
Can you define or explain counting without also referencing/defining/explaining a mathematics concept?
> And it can count fine. It doesn't know how to spell.
It's the other way around, it could spell, it couldn't count the letters.
So it can't spell for garbage because the tokenizer hides the real spellings from the LLM.