Hacker News Reader

Comment by lunarcave | Hacker News Reader

Perhaps the story that doesn't get told more often is how LLMs are changing how humans operate en masse.

When ChatGPT came out, I was increasingly outsourcing my thinking to LMS. It took me a few months to figure out that that's actually harming me - I've lost my ability to think through things a little bit.

The same is true for Coding Assistants; sometimes I disable the in-editor coding suggestions, when I find that my coding has atrophied.

I don't think this is necessarily a bad thing, as long as LMs are ubiquitous and they proliferate throughout society and are extremely reliable and accessible. But they are not there today.

View on HN

People said the same thing about IntelliSense.

The LLM should free you from thinking about relatively unimportant details of programming on the small and allow you to experiment with how things fit together.

If you use the LLM to churn out garbage as is, it's essentially the same as performing random auto-completes with IntelliSense and just going with whatever pops out. Yes, that will hurt you professionally.

Cars free me from the drudgery of walking to places but I still ride my bike to stay fit. You are an energy optimization machine and lots of your biology is use it or lose it.

I look at it more like a lever or pulley. I'm still going to get a good workout exerting the maximum that I can. But wow can I lift a lot more.

Cars are actually a good analogy. They seem helpful in a lot of ways, and they can be, but ultimately they're harmful to your health, other's health, environment, and the design and ability to construct places and locations free of drudgery.

Motorized vehicles enable more access to fresher food, cheaper housing, and more material prosperity. They are genuinely helpful, just like LLMs can do some things insanely good (50x cheaper at categorizing tweets).

They're also cooking the Amazon lol

The power cost of inference and training is coming down exponentially. Deepmind v3 was trained for a tenth of what it cost to train equivalent performing models a few months earlier.

This is like saying digital electronic computers would never be economically and environmentally feasible if we had extrapolated out from the Z3 and ENIAC.

I think we should reserve judgement on the topic.

Estimates on the energy cost of ChatGPT-o3 per request (never mind training) are scary.

Writing some tests then having the chatbot iterate until its code passes the tests is something I should experiment with. Anyone tried that?

...anyone been brave enough to release that code into production?

I do the opposite - I write the code and then ask copilot to create tests, which it gets 50-75% right, and I get to 100%. If I kept asking it to fix the code it'd never get there and would spiral off into greater and ever more verbose nonsense, but it does have its uses as a boilerplate generator.

The continuous deployment folks release into production if all the tests pass. That's what I was thinking about with that comment, really - it's not so much about the origin of the code, it's about tests-as-specification, and how much do we really trust tests?

I agree, in many cases the chatbot would just spiral round and round the problem, even when given the output of the tests. But... has anyone tried it? Even on a toy problem?

Not yet, but it's definitely something worth experimenting.

FWIW, there are precedents to this, e.g. Coq developers typically write the types and (when they're lucky) have "strategies" that write the bodies of functions. This was before LLMs, but now, some people are experimenting with using LLMs for this: https://arxiv.org/abs/2410.19605

And two thousand years ago Socrates said the same thing about writing:

> For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them.

https://www.historyofinformation.com/detail.php?entryid=3894

I love to mention this whenever someone taking the “this new tech is bad” angle. We have to be careful to not just have the bias of “whatever existed when I was born is obvious, anything new that I don’t understand is bad.”

I understand the truth that writing was very much a huge net positive for humanity, but that doesn't mean Socrates was in any way wrong.

To my mind, though, LLMs are just plain inaccurate and I wouldn't want them anywhere near my codebase. I think you gotta keep sharp, and a person who runs well does not walk around with a crutch.

> People said the same thing about IntelliSense.

Personally I think it's true. I do the majority of my coding with IntelliSense off. It has its uses, however flexing the mind and maintaining a clear mental map of how the code is organized is more helpful.

That seems silly to me. I certainly don't to outsource my thinking, but if I type '.fo' and it ghosts up a `.forEach(`, that's just time saved. It also lets me see what properties exist on the object - it's expanding out what I can know in the context of my cursor at any point of time. It saves me a trip to the documentation to see what I can do with an ICollection, but I'm still making the decisions.

I find it invaluable when I have to dive into unfamiliar codebases, which is unfortunately too often. Mental maps are great when you can work with the same code for awhile and keep all of that knowledge in the front of your brain, otherwise you should just outsource them.

I often "feel" like I have all the details in my heads and LLMs effectively makes me type faster. But that's never the case. It's only when I start writing the code myself line by line that I discover blindspots.

For example, in freshman year, I remember reading previous exams solutions (rather than drilling practice problems) to prep for a calculus midterm. It felt good. It felt easy. It felt like I can easily get the answers myself. I barely passed.

My point is LLMs are probably doing more inference/thinking than we give it credit for. If it's making my life easier, it has to be atrophy-ing something right?

I don't entirely disagree with you, but ImtelliSense might have increased the prevalence of long and stupid symbol names. Had people been forced to type out names, their naming might have been more careful.

> I was increasingly outsourcing my thinking to LMS. It took me a few months to figure out that that's actually harming me - I've lost my ability to think through things a little bit.

Been saying this for a while since the early LLM consumer facing products launched. I don't want to stop thinking. If they're helpful as a tool, that's great! But I agree, users need to be mindful about how much thought "outsourcing" they're doing. And it's easy to loose track of that due to how easy it is to just pop open a "chat" imo.

I found that it supercharged parts of my thinking — particularly critical thinking. I just had to read, review and critique so much. And, I had to write very clearly my intent.

But I no longer do manual brainstorming as much. And I am generally overwhelmed by what I might do, I’m more concerned about my time use.

Reply on news.ycombinator.com