Hacker News Reader

Comment by krater23 | Hacker News Reader

This comes some years too late. Who is really using StackOverflow to ask questions? I know that ChatGPT can answer all questions that StackOverflow can answer in a fast, friendly and nearby as correct way. It has limitations, but the most people on StackOverflow are jsut doing the same than ChatGPT, just repeat after other people. All this things will be gone soon.

View on HN

I don't remember where I read this, but will tell this story every time ChatGPT is mentioned as a replacement of StackOverflow.

> I asked chatgpt how do I do X. Chatgpt replied with the accepted answer from stack overflow, thus saving me 1 second. In theory. In practice, what Chatgpt did NOT tell me was that the top voted comment on that accepted answer said "Do not do this, doing X in this way creates a security vulnerability of type Y because of Z"

Honestly, in this scenario I’d lean toward the LLM providing the better answer. Whoever shared this parable didn’t understand that ChatGPT isn’t just a search engine that regurgitates the top voted accepted comment from Stack Overflow. An LLM is probably more likely to parse the entire page and all of the comments if it pulls an URL. It would also be more likely to pull multiple URLs. It would also be more likely to pull from accumulated embedded knowledge. Unfortunately for this fabricated example, the casual visitor to Stack Overflow is most likely to be duped by an incorrect accepted answer if they don’t scroll down and read the fine print (which isn’t that common, a fact that surprises people who always read the comments and all answers).

ChatGPT can provide the same answers SO can because it mined SO as part of its training data.

From my experience, the majority of answers (accepted or highest voted) are wrong/outdated, with the second or third being the correct answer.

Ironically, that's where LLMs seem to have an advantage.

They're not regurgitating the top Stack Overflow answer. They've scraped the entire Stack Overflow page, including context in additional answers, and they've also been trained on the same question from countless other parts of the internet.

An advantage over what? Reading an SO page "scrapes" the whole page with your eyeballs.

Anyway, I didn't say anything about which is preferable to use, I merely commented on where the information comes from, so your response is non sequitur. If sources like SO go away, it will affect the quality of LLM responses. "the same question from countless other parts of the internet" is a figment of your imagination when it comes to the kind of content SO accumulated. I've been in this industry for a very long time (my name appears in RFC #57) and I know what it was like to try to get answers to technical questions before SO showed up.

Then your experience is extremely limited or you're misreporting it.

Or, the context of your use, for a site that covers many thousands, is different than mine, leading to a different experience.

The most-being-wrong applies very reliably for any API, library, or language that isn't stable through time. You probably work C, or similar, where backwards breaking changes, or shifts in "the correct way", are rare. I'm jealous!

The answer being wrong at any point in time is probably < 10%, in my experience.

You've basically confirmed what I said.

Yes. Giant disorganized piles of atomized information are less useful than optimized integrated data representations queryable by LLMs.

As a ConTeXt user I use to ask questions on SO from time to time because (1) the wiki can be outdated or incoherent; and (2) I get lost on the mailing list, which is the other way you can ask stuff or get help about ConTeXt. I don't see ChatGPT or something like that catching up about something so niche fairly soon, though

I still use SO to ask questions. In fact, just asked a question couple days ago which ChatGPT couldn't answer, nor has anyone on SO been able to:

https://stackoverflow.com/questions/79831192/multi-line-titl...

Well, SO is only read by scrapers these days so no answers doesn't mean much. Could be the question is unsolvable, could be lots of people know the answer and won't see it because you've written it in a graveyard.

edit: thought I'd try to check that belief but the process for getting SO to show me any information involves enough layers of question boxes and captions that I've kind of got the confirmation by giving up in annoyance.

Reply on news.ycombinator.com