Have you seen LLM play chess? They would not best an average 6 years old who just learned the rules. I doubt giving them more context would help. There are some thing you can't just parrot your way out.

And I would easily argue that it's harder to copy the behavior of a good software engineer than it is to copy the moves of a good chess players.

Not even talking about understanding the business logic you are actually implementing. Which LLM seems as far as understanding chess.

Well, until Claude Code ends the game(maybe in a few months), yeah. Today we're not there yet, even with full codebase knowledge. At least from my experience and how I want it to be done. For some, the damage has already been done. Wait & see !