Hacker News Reader

Comment by grey-area | Hacker News Reader

There is zero evidence for this.

LLMs make significant mistakes frequently and smart people have no way of judging those mistakes outside their domain expertise. They are also sycophantic and great at being an echo chamber which makes people feel smart even if they are not.

So I think the burden of proof is on you to prove that they somehow amplify intelligence, it seems highly unlikely.

View on HN

Most domains have some kind of internal consistency/theory building you can do. A smart person can certainly notice inconsistencies when trying to learn something. In fact they're likely to be points of confusion that the smart person will dive into just to try to make sense of things, even if they don't suspect the LLM is at fault.

People who think they are smart often think they can get by in any domain without knowledge and context, but at a certain point this fails and knowledge is required, not just intelligence.

Pretty much this. With the internet and my websearch-fu, I can find some facts on a lots of things. And while I do for some questions I have, I try to never interject answers when I don’t have a modicum of mastery in the domain. Mastery is the formation of an internal model that guide your actions and thinking. It’s not merely knowledge and intelligence.

Yes sorry for the shorthand, an internal model is a good way to describe mastery. It’s something LLMs don’t have and can’t transmit.

We're talking about learning here, i.e., building that knowledge and context. People also refer to more than one source, and I think you overstate the degree to which LLMs make mistakes in current year. Human teachers also make mistakes, and it's typical for a bright student who's paying attention to correct even small details.

LLMs are not a shortcut to mastery, in fact the opposite most of the time and their errors are qualitatively different from human errors.

I don't think anyone is claiming they're a shortcut to mastery. Personal tutors aren't a shortcut to mastery either. Both simply have the advantage of being able to deeply know where the student's knowledge frontier is, so they can tailor their answers to that, make connections and draw analogies to things the student has mastered (while pointing out distinctions), etc. The student still needs to put in the work.

LLMs are not tutors and do not deeply know anything.

In the sense that they are not programmers and do not deeply know e.g. performance engineering, sure. But I can still give one a program and ask it to optimize it, and it will turn it into a zero allocation streaming SIMD implementation that's orders of magnitude faster than where it started. It may not "know" how to do any of that, but it does it anyway.

It may or may not do that and it may introduce other problems as it does so. Checking the work is not a trivial problem.

Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?

All of those sound like flaws and defects of dumb people?

> Why would a smart person go to an LLM for an answer they cannot judge or test, be succeptible to flattery and sycophancy rather than picking up on the emotional manipulation and being suspicious/sceptical of the interaction, or looking for support from an echo chamber rather than a Socratic opponent?

Great summary of the flaws with LLM "research"/"reasoning". It's always trying to con you, and I question the literacy and intelligence of the people who can't see this.

Many people think they are smart and are taken in by LLMs. Also many genuinely smart people are unable to judge their competence in other areas (eg Nobel Laureates making pronouncements in fields they know nothing about).

No evidence but somewhat of a counterpoints:

Smart people know LLMs confabulate and tell them they’re Absolutely Right! Smart people don’t want to be embarrassed by trusting the hallucination machine and revealing their gullibility to others.

Smart people are handsome, charming, and always make the right decisions even on things they don’t know about. And if they ever are deceived, well I guess they were no true smart person.

Really widely reported in normie news and on reddit, hallucinations and sycophancy. Stand by that whether you’re a petroleum engineer or a bartender, if you’re clever you were probably either well informed or curious enough not to blindly trust the plagiarism machines.

Reply on news.ycombinator.com