>> I asked ChatGPT to assume "word" is a number, ignore numeric digits (ie 0-9) and assume the characters in "word" are digits. Also assume "word" contains the largest possible digit. What would it conclude the base to be.
For your intended use-case (as explained in your comment) shouldn't you be phrasing the second part of this specification as "treat the characters in "word" as if they are digits"?
The way you worded it introduces some ambiguity. The ambiguity can be resolved by a process of elimination - I had to think of what could possibly happen if I "assumed the characters in 'word' are digits" and realise that it doesn't make sense, so it must mean you want to use the characters as digits.
ChatGPT and LLMs in general really can't be expected to deal with that kind of ambiguity, not least because it's not the entire story (you also need another little logical jump, from "treat characters as digits" to "treat characters as denoting their positional values in the alphabet" or some such).
I'm saying, you're asking ChatGPT to take many reasoning steps, when everyone is having a big debate about whether ChatGPT and LLMs can reason _at all_. To make any heads or tails of ChatGPT's abilities, one needs to start with simple examples, and then get to the more complex stuff.
On the other hand, I expect that if you clarified your question more, or perhaps stated it just the right way, there's a good chance you'd get a correct answer. After all, the idea of counting with characters is very common and it's hard to believe ChatGPT hasn't seen a few thousands of examples of it during training.