Comment by ghaff | Hacker News Reader

Comment by ghaff | Hacker News Reader

Parent Full thread

I was playing around with ChatGPT yesterday and what I found was:

- Ask a straightforward question like what are business models for open source software or what are the advantages of GPL vs. AGPL vs. MIT and the answers were pretty shallow and boilerplate but they were correct as far as they went.

- But ask a question like "Why didn't large SMP systems become more mainstream?" (which basically requires synthesizing a lot of knowledge about the last 25 years of computing history) and it spewed out an explanation that basically got enough right to be dangerous and got at least one thing pretty much entirely wrong. But it was coherent enough that someone who didn't have any background would probably think their question was answered.

Also, if you’d ask it “Why did large SMP systems become mainstream?", it would (presumably) spit out a plausible-sounding explanation for that. The models seem to take the prompt as a premise, and then go from there. Maybe that’s also why when you point out a mistake, they oblige and provide a correction.

I haven’t tried: If you point out something they said as a mistake when in fact it was correct, do they notice and object? Maybe if the evidence from the training data is strong enough.

Interesting. I'll have to try that. And, yes, you are correct. It gave a short pablum answer that pretty much focused on the benefits of multiple cores.

The mistake in the original was somewhat subtle and gets into programming for large SMP vs. MPP/distributed systems. And it's actually true that parallel programming even on shared memory systems has challenges (and there was a lot of discussion at one point when multicore was coming in on desktops) but it's pretty much flat wrong to say that we didn't have large SMP systems primarily because they were hard to program for.

See https://news.ycombinator.com/item?id=33844574 where the model acknowledges a non-existing mistake after prompting.

I did ask both questions and found the answer to "Why did large SMP systems become mainstream?" more coherent. Broken down into bullet points, the arguments given for both questions would be AFAIK correct.

Reply on news.ycombinator.com