a lot of people trying to get a language model to do maths and coding. obviously this is not actually within its capabilities to properly reason about...

You can ask it non-trivial things about maths (from a beginner at maths perspective....) and it will just fabricate nonsense..

like 'find the missing number' for: 5,10,?,50,122

It said this:

starting with 5, add 5 to get 10 starting with 10, add 10 to get 'x' starting with 'x' add 40 to get 50 starting with 50, add 72 to get 122

then it goes on to say 'using this pattern we can find the value of x by subtracting 10 from the third term.

x = 10 + (50-10)/2 x = 10 + 20 x = 30

Amazing, complete and utter nonsense. The real answer of course is a bit difficult here for normal people like me..

But any person can clearly see that even given its own logic, it still violates its own logic...

x + 40 would be 50 in its initial logic. 30 + 40 is clearly not 50... - if it understood _anything_ about numbers and values, it would never have said such a thing.

This also clearly shows it refuses to give you an 'i dont know', and would rather spout out complete and utter nonsense than a real answer....