I’ve mentioned it before, but we can get some pretty good results for developing implementation and tests with this setup. The context isolation makes it easy to separate test and code writing using a specification as a source of truth. Tests are then run by an arbiter that feedback to tests or implementation depending on who it blames for test failures. The tests are then treated as an independent implementation, and through the math of coincident errors, the arbiter feedback loop ensures that both are eventually correct (N-versions and clean room research did this with humans in the 80s). This then allows us to use less reliable LLMs to develop code, like Qwen on a MacBook Pro.
I hope to have something out on GitHub before my beach period ends (well, if I’m unsuccessful in getting a new role, then I’ll have a lot more time to play at least).