Yes, they’re very good coders; no, they aren’t perfect. Sometimes they make trivial mistakes or hallucinate, and other times they have unique insights about difficult problems or one-shot a lot of complex tasks.
If you don’t see the value of AI even at this point, you’re either lacking imagination and/or just too stuck in your own ways.
For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.
It seems that way if you judge them the same way you would human coders, but they’re different. They might be able to do things that veteran coders can’t without spending days on it, and fail at things that beginners can do in half an hour.
A car might not be able to traverse difficult terrain as well as a horse, but it doesn’t mean the car is not a good mode of transportation.
Also, their eagerness to produce. A good employee (senior or not!), knows when they're at the end of their wits and say so. In areas where LLMs are knowledgeable, but not knowledgeable enough, they'll still produce. They're stuck at unconscious incompetence, kind of like a systematic Dunning-Kruger effect. To solve that, we can either increase competence (moving out the area where this effect is noticeable), or maybe introduce some kind of feedback, self reflection (one could say, some kind of conciousness [0]).
[0] inspired by "Goedel, Escher, Bach"
That doesn’t mean they are unpredictably bad, it means assuming that they behave like humans is a bad way of predicting their behaviour. “Unlike humans” and “unpredictable” are not the same thing. If you spend time working with them, you get a better sense of what they are good and bad at and get better at predicting their behaviour.
The sibling comment says it better - it's about predicability of mistakes, and the effort and knowledge necessary to correct them.
Maybe the next generation of models will solve these problems, who knows.
So if you drive well and can control this "car", you are able to fix that and benefit from the fast "car" a lot.
> or a car that randomly refuses to go on certain roads
This doesn't seem like a good comparison. AI will do anything, it may just be wrong.
I'm fine with ditching the car analogy though.
Anecdotally, I think this behavior is undesirable for most commercial LLM use cases. I have several friends that have complained about Gemini’s “back talking” and prefer ChatGPT’s relative sycophancy.
If you tried to replace a programmer with whatever LLM it would completely flop. Just write a script to fetch Jira tickets and let it rip - it won't work well. Because it doesn't know that maybe the Jira ticket is stupid, or that if they implement this change it requires insight from Team X, or that this bug is really intended behavior, etc.
Being a programmer is much more than, like, making a neat regular expression or generating slightly customized boilerplate. As it stands, these are assistants in which programmers guide and handhold them the whole way. They are not coders.
Making those decisions and communicating is a big chunk of being a programmer.
> bu-but you can waste your day writing prompts to get around the fact that LLMs are brainless!
Yes, and?