The amusing things LLMs do when they have been at a problem for some time and cannot fix it:
- Removing problematic tests altogether
- Making up libs
- Providing a stub and asking you to fill in the code
- Removing problematic tests altogether
- Making up libs
- Providing a stub and asking you to fill in the code
This is an attack vector. Probe the models for commonly hallucinated libraries (on npm or github or wherever) and then go and create those libraries with malicious code.
This is a perennial issue in chatbot-style apps, but I've never had it happen in Claude Code.