I think it's key to not give it contradictory instructions, which is an easy mistake to make if you forget where you started.

As an example, I know of an instance where the LLM claimed it had tried a test on its laptop. This obviously isn't true so the user argued with it. But they'd originally told it that it was a Senior Software Engineer so playing that role, saying you tested locally is fine.

As soon as you start arguing with those minor points you break the context; now it's both a Software Engineer and an LLM. Of course you get confused responses if you do that.