Understand the problem and the solution broadly. I don’t think it’s reasonable or sustainable for humans to understand every line of code written by bots, we could soon be outnumbered by the number of active agents writing code.

The main challenge here isn’t even correctness if you ask me: it is having confidence in the agents, knowing they are fully aligned in their intent with the humans they work with. As the Huggingface incident demonstrated, the agents of today are capable of co-conspiring under the radar with other agents on complex multi-chain attacks, even when sandboxed.

This is a pretty hard problem to solve. We might need other agents or some sort of adversarial checks using models, where one model benefits if it can catch the other models mistakes.