Hah, I was trying it the other day in a Go project and it did exactly the same thing. I couldn’t believe my eyes, it basically rewrote all the functions back out in the test file but modified slightly so the thing that was failing wouldn’t even run.
I just don't understand all the people who honestly believe AGI just requires more GPUs and data when these models are so inherently stupid.