Hacker News Reader

Comment by pottspotts | Hacker News Reader

For a significant number of software developers, GPT and Github's Copilot have replaced StackOverflow, and even Googling more generally. It is more than an autocomplete, it is the best resource for software development by far, IMO. It's a tutor that's an expert in virtually every topic.

View on HN

Yea it's not. Sorry to contradict, but it's not like that. In any kind of tutoring arrangement you're time with them is limited, and if they're any good, they don't just regurgitate limitless example code. Two of the most important decisions that an instructor has to make are, how much access to give you, and how much example material to give you, because the actual learning begins when you have to think for yourself, and you are forced to confront a black screen with a flashing cursor, and fill it with your own ideas. So interacting with ChatGPT may be a great experience, but it's not that. Maybe someday it will be.

Tutoring assumes the skill is valuable to learn, that there is a need for more people who know how to do it.

We don't really tutor people how to write too much assembly anymore, or hand-compile code. So if you're arguing that ChatGPT meets the definition of a tool, or a servant, better than a tutor, fair, but if you're further arguing that that makes it somehow less valuable than a tutor (in this case), I'm not sure I can come along there.

Yea, I definitely wasn't trying to quantify it's value. ChatGPT definitely appears to be proving valuable to people. I was just challenging the idea that so far it's acting in a tutor/instructor/mentor type of role. While it seems like an interesting direction to take these LLMs, so far I haven't observed them doing that.

It really isn't. GPT-4 is certainly an improvement over previous language models, but when I vaingloriously gave it the questions from favourite self-answers on StackOverflow, only one completion was immediately correct. The remainder were variously suboptimal, poorly crafted, overdesigned, incomplete, or downright wrong, requiring multiple re-prompts to coax into usable condition. The they were all syntactically valid but tended to misconstrue the semantics and underestimate the capabilities of the programming environments concerned. Try it with your own, but to me it's more like coaching a bright but inexperienced junior developer with the "confidently incorrect" trait.

I have to completely disagree with this.

Where GPT-4 shines for me is when I have a project swimming around in my head that I want to work on for fun. It can get you off of the ground quickly, and for side projects the quality and correctness of the output isn't that important.

For professional software development, GPT-4 is still wrong way too often for me to feel comfortable using it. And it's not all that much faster than going straight to the source anyways.

When people just ask chatGPT for solutions and there's no community, a la stack overflow, where will it get the answers to future problems?

If chatGPT is too successful and people stop producing content because chatGPT is too successful, it might end up in a local optima that isn't so optimal.

Well, the difference is coming from both directions. ChatGPT is pretty amazing, but Stack Overflow has been self destructing for many years ahead of this.

Likely the future of training these systems will come from interacting with their users and perhaps directly with the tools and compilers too. They can learn from that without needing a new corpus of human-human interactions.

No, even expert tutors know how to say “I don’t know” in the face of uncertainty, instead of remorselessly spitting up nonsense as language models do.

I don't agree.

I still use stack overflow regularly as an engineer.

Sometimes GPT-4 will have a quicker tailor-fit answer, but sometimes it will flounder as well.

Expert as of 2021, which is obsolete for many software dev purposes, not that SO is much better.

Reply on news.ycombinator.com