They're great at some stuff and terrible at other stuff in ways that are very hard to predict.
I'm figuring out new and better ways to use them in a daily basis, and I've been an almost daily user for nearly three years.
Let's say models can exactly and correctly write any code you ask of them.
- How do you break down a project into a sequence of requests to models?
- How can you most effectively parallelize the work - models will never be instant, so there will always be benefits in working out how best to use several agents at once
- Now that the models can handle the details of Lean, and Swift-UI, and Oracle stored procedures, and thousands of other technologies that you never got around to learning in the past... what can you do with those and how do you pick which projects to go after?
- How do you collaborate with other engineers and designers and product people in a world where you can churn out the right code reliably in a few minutes?
The models we have today are already effective enough to change the shape of our work as software engineers. As the models continue to improve figuring out and adapting to whatever that new shape is becomes even more complicated.
You need to be able to estimate how long an approach is likely to take in order to correctly prioritize your work.
Coding agents throw a wrench in this because some of the stuff that used to take a long time doesn't take a long time any more. I'm finding my 25+ years of experience in estimating software has been completely thrown off - it's taking a whole lot of work for me to start building up those intuitions about what's quick and what takes a long time.
With coding agents you need to think very carefully about how you design the agentic loop such that the agent has the right tools and information available to it to compete the goal.
I've been writing a lot more about that here: https://simonwillison.net/guides/agentic-engineering-pattern...