The quirky thing to remember about gpt-3 is that it really is just a giant autocomplete based on the internet. It can do math insofar as it’s memorized some text which did that math with slightly different verbiage etc.

If you ask it to compute something that would never have been seen on the internet it’s likely to fail. E.g. add 2 extremely large/rare numbers together