The LLM should free you from thinking about relatively unimportant details of programming on the small and allow you to experiment with how things fit together.
If you use the LLM to churn out garbage as is, it's essentially the same as performing random auto-completes with IntelliSense and just going with whatever pops out. Yes, that will hurt you professionally.
> For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them.
https://www.historyofinformation.com/detail.php?entryid=3894
I love to mention this whenever someone taking the “this new tech is bad” angle. We have to be careful to not just have the bias of “whatever existed when I was born is obvious, anything new that I don’t understand is bad.”
To my mind, though, LLMs are just plain inaccurate and I wouldn't want them anywhere near my codebase. I think you gotta keep sharp, and a person who runs well does not walk around with a crutch.
For example, in freshman year, I remember reading previous exams solutions (rather than drilling practice problems) to prep for a calculus midterm. It felt good. It felt easy. It felt like I can easily get the answers myself. I barely passed.
My point is LLMs are probably doing more inference/thinking than we give it credit for. If it's making my life easier, it has to be atrophy-ing something right?
Personally I think it's true. I do the majority of my coding with IntelliSense off. It has its uses, however flexing the mind and maintaining a clear mental map of how the code is organized is more helpful.
This is like saying digital electronic computers would never be economically and environmentally feasible if we had extrapolated out from the Z3 and ENIAC.
Estimates on the energy cost of ChatGPT-o3 per request (never mind training) are scary.
...anyone been brave enough to release that code into production?
I agree, in many cases the chatbot would just spiral round and round the problem, even when given the output of the tests. But... has anyone tried it? Even on a toy problem?
FWIW, there are precedents to this, e.g. Coq developers typically write the types and (when they're lucky) have "strategies" that write the bodies of functions. This was before LLMs, but now, some people are experimenting with using LLMs for this: https://arxiv.org/abs/2410.19605