We Fuzz All the Things – Using Fuzzing to Improve Our Agent's Capabilities

Hey Ian from Bismuth here, we see a lot of people talking about how "dumb" LLMs are and one thing I think people are still lagging behind on is integrating advanced code analysis into the agent loop.

For inner loop tools like Cursor this means using better static analysis and maybe test running.

For tools like ours on the outer loop and asynchronous you can lean on more advanced techniques like fuzzing to find bugs and use them as part of the agent loop to produce a higher quality artifact.

It definitely may take 5-10 minutes of extra time, but that isn't a problem when you can take an 20-40 minutes to finish a task.

So I think as a young industry we've barely scratched the tip of the iceberg on techniques to guide code generation and prevent unwanted outcomes for developers.

We've been using this very effectively as we've rolled out with early design partners and we also have some other content coming up very soon that cements the usefulness of this technique.