Then I asked it to modify it by eliminating one of the constraints on the problem. It did a very convincing "Ah, if we need [that] we need to do [this] and output a new version... that didn't actually work right.
I pointed out the specific edge case, it said "you are correct, for that sort of case we have to modify it" and then spit out exactly the same code as the last attempt.
The most interesting thing to me there isn't that it got it wrong - it's that spitting out exactly the same output without realizing it, while saying that you are going to do something different, is the clearest demonstration I've seen from it that it doesn't "understand" in human-like ways.
Extremely powerful and useful, but VERY important for users to know where it runs into the wall. Since it often won't tell you on its own.