My disillusionment comes from the feeling I am just cosplaying my job. There is nothing to distinguish one cosplayer from another. I am just doordashing software, at this point, and I'm not in control.
I think the way you’re using these tools that makes you feel this way is a choice. You’re choosing to not be in control and do as little as possible.
We won't know until the code being produced especially greenfields hits any kind of maturity 5 years+ atleast?
If you're using LLM's and you don't know what good/bad output looks like then of course you're going to have problems, but such a person would have the same problems without the LLM...
That's what it's ultimately been tuned to do.
The way I see this play out is output that satisfied me but that I would not produce myself.
Over a large project that adds up and typically is glaringly obvious to everyone but the person who was using the LLM.
My only guess as to why that is, is because most of what we do and why we do it we're not conscious of. The threshold we'd intervene at is higher than the original effort it takes to do the right thing.
If these things don't apply to you. Then I think you're coming up to a golden era.
It's like a junior dev writing features for a product everyday vs a principle engineer. The junior might be adding a feature with O(n^2) performance while principle has seen this before and writes it O(log n).
If the feature never reaches significance, the "better" solution doesn't matter, but it might!
The principle may write once and it is solid and never touched, but the junior might be good enough to never need coming back to, same with a llm and the right operator.
What they're worse at is the bits I can't easily see.
An example is that I recently was working on a project building a library with Claude. The code in pieces all looked excellent.
When I wrote some code making use of it several similar functions which were conceptually similar had signatures that were subtly mismatched.
Different programmers might have picked each patterns. And probably consistently made similar rules for the various projects they worked on.
To an LLM they are just happenstances and feel no friction.
A real project with real humans writing the code would notice the mismatch. Even if they aren't working on those parts at the same time just from working on it across say a weekend.
But how many more decisions do we make convenient only for us meat bags that a LLM doesn't notice?
Better yet, go up one level and and think about how to avoid the other classifications of problems you don't know about, how can the LLM catch these before it writes the code... etc.
Once you start using it intelligently, the results can be really satisfying and helpful. People complaining about 1000 lines of codes being generated? Ask it to generate functions one at a time and make small implementations. People complaining about having to run a linter? Ask it to automatically run it after each code execution. People complaining about losing track? Have it log every modifications in a file.
I think you get my point. You need to treat it as a super powerful tool that can do so many things that you have to guide it if you want to have a result that conforms to what you have in mind.
They are amazing for side projects but not for serious code with real world impact where most of the context is in multiple people's head.