Hacker News Reader

Comment by aimazon | Hacker News Reader

I don’t think you’re a dimwit but I read your post[1] with an example and I am curious to whether you feel you’re losing something by telling the LLM it’s wrong and to try again, rather than going through the exploratory/iterative learning process yourself. For example, would you have known to ask about GeoJSON if you had not come across and learned about it pre-LLM? More succinctly: do you feel you’re learning more or less or an equal amount when using LLMs?

[1] https://simonwillison.net/2024/Mar/22/claude-and-chatgpt-cas...

View on HN

Absolutely I could have learned more from that particular project if I'd spent more time with it rather than getting the LLM to do the work... but that's why I like it as an example: since it was effectively a distraction (a "side quest") the alternative wasn't learning more, it was not doing it at all (and learning nothing).

I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them.

I do feel like my rate of learning has increased though, because I'm much more likely to try out a completely new technology when I know an LLM can flatten the learning curve for me a bit.

> I do feel like my rate of learning has increased

It's the same for me. Lots of experience knowing what to ask for. It does a better job in summarizing knowledge and getting me a relatively coherent explanation. Much faster than using Google Search to find and connect the dots from dozens of pages.

I just don't see much benefit in its reasoning and code assistance features besides basic stuff.

> I have 20+ years of experience helping me know what questions to ask of them.

That seems rather unlikely given that LLMs didn't exist more than seven years ago, much less 20+.

They mean they have 20 years experience in programming, which helps them ask good questions.

Yeah, that's what I meant. With LLMs having deep domain knowledge in a field helps enormously in terms of prompting them as productively as possible.

So, to maximize the usefulness of LLMs, requires expert level domain knowledge in a field? Sounds like a useful tool for highly trained experts. For the average Joe Developer, perhaps not.

> I'm able to get really great results out of LLMs because I have 20+ years of experience helping me know what questions to ask of them.

While a direct answer is nice, I like an iterative/explorative process because of all the things I pick up alongside it. An example is when I was working on an epub reader for macOS (side project). I wanted to a native layout engine instead of a webview and I decided to go with muPDF. This has lead me to know more about text layout and rendering, and embedding C inside Swift than I would if I just have direct answers for every problem (if I'd know the correct questions in the first place).

I accumulate side quests like that until I can do a nice experiment to learn as much as I can for a particular domain space.

You're talking right past them though: you were working on a side quest you had interest and bandwidth to iterate on.

I agree with the parent comment, my pipeline is already saturated with side quests, I'm already iterating on a bunch of random fun work, and side-project work and work work. So often times the most "LLM-heavy" projects of mine are things I straight up would not do if I didn't have something to get the ball rolling other than more of my own free time which is already in short supply.

Hopping from direct answer to direct answer isn't where I find wonder/fun in programming anyways, but sometimes you don't have bandwidth for the side quest to be fun or wonderous.

I was trying to fetch key-value pairs out of a database using PHP+PDO the other day and I knew there was a nice easy to do it but I couldn't remember how. Something about fetchAll, maybe PDO::FETCH_GROUP|PDO::FETCH_COLUMN.... what was it?

So I asked a couple LLMs. They wrote out loops for me to format the data how I wanted. I could have copy-and-pasted that in and it would probably have worked. But I felt there was something better yet, so back to Google I go.

It's `PDO::FETCH_KEY_PAIR`. It's built-in. But oddly kind of hard to find unless you know the right thing to search for, and "key pair" was not springing to my mind.

Point is, if you just let the LLMs do your work you won't even find these better ways of doing things. And I'm quite afraid of what the LLMs are going to do to Google, Stackoverflow and documentation in general. In a couple users it'll be ungoogleable too.

As noted LLMs can only give you what you ask for, but for a lot of problems what you ask for isn't what you need; it's two or three steps removed. And LLMs can't tell you that you're doing something wrong; unlike curmudgeonly users on SO or in various forums/channels.

My gut feeling is that we're going to enter into a 'dark age' of coding where a lot of previously available resources are going to be ransacked and made hard to find in favor of big corporation owned LLMs. It's already having an extremely bad effect on search in general; we're potentially only a few fights away from sites like SO having users leave en masse. That's why I think having a strong network of engineers to talk with will become more important than ever, almost a return to the IRC days.

Strong disagree. I have learned so many new things since I started using LLM's. Some of them I'm actually embarrassed to admit, because I should have known about them a decade ago.

If you work in a small company and you are the most experienced developer, you don't often get feedback on how you can improve things.

The trick is, quite simply: just ask. I regularly dump some code I wrote in a language model and then ask what can be done better.

I would never do that in any online space, because first, I don't wait an answer maybe some day, I need an answer NOW. And second, I prefer to avoid being called a fool.

This is precisely the wrong way to engage with LLMs. If you are asking it 'what can be done better', it'll spit out something. That something isn't necessarily better or not because it has no concept of 'better' or 'worse'.

Ah, so that's why all my code has become more concise and efficient and I've learned countless new tricks that I did not know before and probably would have never found without LLM's.

Too bad I'm "engaging with them wrong", I could have sworn it was helping me.

Seriously though, claiming LLM's don't have any higher level understanding of right and wrong and then extrapolating that to "they cannot possibly be used to improve things" is a very stubborn refusal of the fact that the most logical answer to the question "what can be improved here" is... actual improvements.

They do not have any higher level understanding of right and wrong. You lead the model on by telling it to improve something, so it will rework the code in question and tell you that it's an improvement. Regardless or not if it is. Coding is about 70% subjective, 30% objective when it comes to figuring out improvements because the majority of improvements deal with business logic and things specific to your domain.

You seem to be writing from a lost world when people enjoyed coding because they enjoyed learning new things with every project. I think that's sadly something only enjoyed at work by a very small minority these days.

I very much aim to learn something new with every project. The thing I like most about working with LLMs is that they let me take on MORE projects!

I don't think those 'good old days' ever existed. Or rather, the situation hasn't really gotten worse.

I probably wasn't clear I still live in that world, both professionally and in personal projects. But my perception is that the vast majority of devs and engineers do not.

Reply on news.ycombinator.com