Hacker News Reader

Comment by 3x3m3 | Hacker News Reader

Nothing made me feel as dumb as reading its chain of thought. It made it click for me how hopeless we will be soon, once we cannot comprehend even the reasoning for some things.

View on HN

I'm not so sure. LLMs are more like human simulators and are pulling this "reasoning" from what they've seen in the training data. It therefore might be possible that they cannot be more intelligent than those who wrote the source material.

Let an expert in the field examine its reasoning first. LLMs are great at putting out reasonable sounding bullshit.

Just because it came up with a roughly equal result doesn’t mean its explanation is sound.

This is not a wrong sentiment. I'd be super interested in hearing an analysis of its CoT. To my ears it was a nearly schizophrenic roundabout with a lot of contextual things like "that's high" but I don't know why it thinks that.

What I think is incredible about this example is how adept it is at mixing precise calculations with rough heuristics. That's exactly what I was taught, to validate the numbers by asking "Does this answer seem reasonable?"

ChatGPT doesn't do that at all, as far as I'm aware

“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”

- Frank Herbert, Dune

It's sad to say, but I would indeed trust my life more to a Harkonnen than a machine.

Reply on news.ycombinator.com