I have the same experience with the test I give my back-end devs. ChatGPT can't even begin to decode an encoded string if you don't tell it which encoding was used.

ChatGPT is great at some well defined, already solved problems. But once you get to the messy real world, the wheels come off.