Of course there are errors and gaffes and weird bits all over, but that helps keep me in the loop and understanding it better.
I can now (more) boldly go where I feared to tread before.
I've been working software development for close to 40 years now. I've worked in plenty of languages, software stacks, ecosystems, from assembler to VAXen to Unix (pre-dating Linux), to cloud stacks to...well, a lot of systems.
The software ecosystem has ballooned so tremendously in that time frame that no human can really be an expert in all of them, or even in a tiny subset of all of them. As I mentioned in another comment, I couldn't be bothered to keep up with the Javascript/Typescript/Web front end drama. I don't do rollercoasters, and that ride is enough to make even the staunchest stomach heave with distress from the endless loop-de-loops.
With LLMs, I can describe a little problem unique to my problem space, inspect the output, research it and even reverse engineer it, and see how well it fits (or doesn't). This is not unlike being a software engineering manager leading teams.
When I said being able to see the results was "cool", I did not mean or imply it was about me trying to "look cool". In fact, I really don't know where that is coming from. I do this for my day job and also on the side for a lot of different causes and personal projects, and no one could give a rats ass about how "cool" I am. Seriously.
People need to stop taking this all so personally. It's a tool. Maybe you all need to step back to my world in 1981 when I first started playing with code as a kid, and compare it to 2016. The advances from '81 to '16 are probably far more dramatic than what AI is doing now.
As for 'looking cool', I meant the code with its fancy formats and highlighting, no snark intended :)
LLMs are a great tool for building, especially for those who could conceivably be engineer team leads/project managers. For greenfield learning, its sub-par unless its guided well.
I have found the LLMs work best where I have some knowledge already, and I can spot the gaffes and mistakes more quickly. It has to be rough on people flying them blind, as it were.
Unrelated but I have tried out LLMs on some legal research, and it has been appallingly bad. The LLMs get the gist of a lot of it, but falls down completely because the legal realm requires a certain amount of exactitude. Time and time again I will ask about laws in a certain U.S. State, and it will show me a law from Arizona or something which has no or limited applicability (you can try to cross reference other State laws in certain State courts, but usually that doesn't go very far unless it is a very novel argument).
You hit the nail on the head with the fancy outputs, I am guilty as charged there. The LLMs are frankly better at it then I am.
Writing things down and then arguing with an agent that constantly gets things wrong is just a new kind of Rubber Duck.
Are Rubber Ducks offloading thinking?
These are more like search plus memory tools.
Memory loss is real but that has been happening for me way before agents.
Depends on your usage. A rubber duck won't upheave the entire history of written human knowledge to reach for your answer. It doesn't have inherent bias, and its purpose could also be handled by a wall, or the air infront of you if you can stretch your mind a little. What I'm going for here is that as you said, its just a hack to make your brain look for answers in a more efficient manner, by clearly underlining your thought process out loud, tricking yourself into conversation mode, yada yada. Its still you doing all the work, you're just being smarter about it.
A LLM is more akin to a senior pair programmer. It may seem like a rubber duck on the surface, in the sense that you're having a conversation, but the senior programmer may come up with ideas you wouldn't think of. They may have more innate experience in the domain which might pull you away to very different implementations from what you had in mind initially. This is all good, but a common pitfall with LLMs is that they're much more conformative than a senior engineer, much faster, much more responsive. This is where it gets dangerous: they don't tell you you're wrong 80% of the time, like a rubebr duck. They also have innate experience of a domain, like a senior engineer. The result is that your brain never gets a 'hang' signal, and it doesn't feel the need to reevaluate.
I don't know, it seems more like every question goes to a contract agency, where a random programmer answers it. Generally they are very good at bringing up relevant things you've never heard of, but depending on who gets assigned to that answer, the implementation might be copy-pastable into production, or it might be a questionable hack, or it might be workable but clearly not the right approach if you're looking for those things. The agency must work by the hour, though, because if you challenge the assertions of the previous guy, the next guy will obsequiously say that you are, of course, quite right and possibly insightful and spit out a new solution more along the lines of the new guidance.
Just that an actual rubber duck doesn’t do anything. You solve the problem you have by talking, and in doing so, thinking, to come up with a solution, an idea, or gain better understanding.
After that you either implement something yourself or have learned something.
With an LLM you offload all of that, the only thing you still do is tell it what the problem is. The agentic duck does the rest and you look at the output.
Even if you have to argue, you argue without having gone through the steps to gain anything yourself.
To be clear, I am closing in on 60 years old. I’ve written a lot of code in a lot of languages. I have not bothered to check every fad that has come down in front end web and Javascript for the past umpteen years. Front end hasn’t been my thing. With current AI, I can leap frog a lot of bull crap I don’t care about and see what at least the AI thinks is good. I still retain a brain, I can still investigate what it did and form an opinion about it.
I treat the AI stuff as idiot savants that can make really nice stuff or go totally off in the weeds. But it is more than good enough to help me be a better me at what I do.
Uh, have you asked it?
These sorts of articles are funny. Like the author assumed that you have to use an LLM in a certain way.
“I can just ask the LLM and there it is! I didn’t learn anything! :(“
You can interrogate the LLM. You can ask it all sorts of things. Tell it to justify its decisions. Tell it to give you supporting evidence. Tell it to give you options. Tell it to explain things in detail.
I’ve been using it as a learning tool and it’s quite effective.
Granted the stuff I don’t care about really does get worse. If I have a config typo I just go “There’s a typo in blah. Find it and fix it.” Or setting up the boiler plate for a project. I’m getting quite useless at that.
But it doesn’t have to be any particular way. It’s a tool. Exploit it.
Hopefully you realize that it's not actually introspecting into its previous thought process, and instead it is just making up a plausible after-the-fact explanation that matches the format of similar explanations found in its training set?
I too often find these explanations enlightening, but please don't forget that you need to read all such responses as those of a pathologically lying psychopath desparate to trick you, not as a trusted expert who actually understands its internals.
If you start blindly accepting everything it says, good luck to you.
But you can do things like get one agent to do something. Spin up another agent to review and critique. Use different LLMs to critique. I’ve had the same agent write piece of code and then come back and say “there are bugs x, y and z”.
For most code, it’s going to do a fairly good job and tell you best practices, give you things to research by yourself.
But if I end up feeding it JIRA tickets, playing agent management, and being QA on the other side, that's really not the job I signed up for.
It helps me pickup and understand new things pretty fast.
And when I need to learn something new I hate doing courses because they're always too slowly paced for me. All courses are paced for the slowest possible participant. Especially those from the US where trainers have this "training voice" where they speak slowly and articulated and usually non-skippable which makes me scream in agony. My brain can't slow down to that level, I just lose focus then. It's like driving a ferrari at 5km/hr. Especially the non-IT courses are so slow and constantly repeat everything, as if I didn't fully understand them the first time, argh. So I tend to spend my time on these awfully designed mandatory HR courses trying to fuck around in developer mode and re-enable the skip segment features :) The more boring the training the more I learn about moodle and its frameworks.
I just love to dive right in and just do something but I often skip that first analysis of what method would be best. And I tend to stick with what I know. Which is often not the best way in a greenfield project. And often the boring first steps are so awful for me that I just put it off. If I do start I often do it half-assed. Then I run into a wall, have to regroup etc. I learn from that (mistakes are always a learing experience), but it's slow going because I take the wrong direction first.
AI really helps me get past that first step meaning I actually start it instead of procrastinating the research forever and move onto the next thing. And I can do great things because I know what I want and how I want it. It's a real enabler for me especially something like cowork. It helps me widen my horizon efficiently instead of wasting my time and energy like online courses do. It picks up things that my weak points are (patience) and feeds me information in a highly dense format I can easily digest. Helped by custom prompts of course ("don't tell me I make great decisions, evaluate everything on its merits, just tell me the meat of the content, don't elaborate"). I still work along with it and take ownership of its output, I'm way too much of a control freak to let the AI do everything without understanding it. But it gets me there quicker and it avoids procrastination by skipping the boring steps. I ask it to suggest methods and techniques, compare their pros and cons so I fully understand what I'm selecting for.
I have to say this was not really the case with early simple question/answer LLMs. The hallucinations were too bad, the searches too limited. The knowledge too hit and miss. It would come up with wrong stuff or overlook the best options. But the new high-context agentic technologies like cowork that really investigate the source material for the purposes I provide, they really are a game changer for me.
For example, I recently asked it about an obscure error I'd been having. Previous simple LLMs would just collapse into a directionless troubleshooting mode, "Did you try...?". Cowork actually went and retrieved all the relevant code from github without being asked to do so, and evaluated what would have happened in the background for that message to appear. Really good to see because it's exactly what I would have done. But I would have taken much more time to do it.
So no, for me it's not making me lose savviness. It's helping me grow it in exactly the way that's efficient for me.
With AI, I can describe my exact use case and see what it gives me. Sometimes it will just cough up a furball like a boilerplate example, but usually it gets pretty close, and I can see how my (maybe weird) edge case fits into the framework or API or whatever.
It may be that I approach problems in a weird way, and that is why examples are less helpful than with some, but AI generally does better here for me.
I guess what you’re saying is you like reading fully fledged code, almost like an apprentice. I would never think to delegate that responsibility to a next word guesser.
Just curious. No right or wrong answer possible.