Hacker News Reader

Comment by DougN7 | Hacker News Reader

Considering hallucinations, that seems risky. How do you double check what you were taught?

Don't ask them to teach you, ask them to make a self-study syllabus/roadmap with online references. It's likely that it ingested the work of others in exactly this scenario, so it shouldn't confabulate as easily.

The same way you double check with any other method you prefer? Duh.

LLMs are vastly superior to compile and spread knowledge than any other thing preceding them.

You double check every university lecture you've been apart of?

That's what is called "studying" or "reading a textbook", isn't it?

Uhm no? Reading a textbook is obviously not the same as fact checking a textbook.

Parent was writing about a university LECTURE which is different from a TEXTBOOK (which is different from primary sources), so yeah, consulting other sources is checking the facts.

Oh I see what you're saying. It was slightly ambiguous.

But in any case, I didn't read a single textbook at uni; it was all lecture notes provided by the lecturers (fill-in-the-gaps actually which worked waaaay better than you'd think). So the answer is still no - I didn't fact check them and I didn't need to because they didn't wildly hallucinate like AI does.

The real answer is:

You should have a mental model about how the world works and the fundamental rules of the context where you're operating. Even though you might not know something, you eventually develop an intuition of what makes sense and what doesn't. And yes, that applies even to "university lectures" since a lot of professors make mistakes/are wrong plenty of times.

Taking an LLM's output at face value would be dumb, yes. But it would be equally dumb to take only what's written on a book at face value, or a YouTube video, or anyone you listen to. You have to dig in, you have to do the homework.

LLMs make it much easier for you to do this homework. Sure, they still make mistakes, but they get you 90% of the way in minutes(!) and almost for free.

I don't think it's (necessarily) equally dumb. Maybe if comparing LLM output to a book chosen at random. But I would feel much safer taking a passage from Knuth at face value than a comparable LLM passage on algorithms.

Did you just sit there in class and then never do anything with what you learned afterwards? That certainly isn't how I approached university.

Doing something with the knowledge given in a lecture is very distinct from fact checking it

I'd say it's a subset of fact checking it. You can check facts without doing anything else, but doing something with the knowledge is inherently checking it. If the lecture presents some programming technique, and I implement it, I'll find out pretty quickly if it's wrong.

They are faster, but I don't see how they are vastly superior to a course designed and offered by a subject matter expert in the field.

they have been trained on material not just by single subject matter expert but all of them :)

They have not, because a large portion of the knowledge obtained by subject matter experts in any given field has never been published.

Also, hallucinations are still a thing, and there's a reason why LLMs do not outperform subject matter experts in nearly every field.

I was being facetious but am now extremely curious about large portion of the knowledge obtained by subject matter experts in any given field has never been published - this is not only strange to me in the small but you are claiming that this is large portion so I am wondering if you have any example(s) to share?

Ah, sorry I missed that.

Academics, whose entire careers are based on publishing knowledge, only publish a fraction of their total knowledge obtained over their career. Estimates are that only 10-20% of all knowledge is explicit.

You can't beat a Caltech-tier lecture, for sure. But you know many people have access to that? You do know. Thousands, and I'm being generous.

LLMs level the playing field for the other 8 billion people.

Reminds of this article[1] that was featured yesterday and which I think was great!

1: https://news.ycombinator.com/item?id=46254794

In addition to the content available on the platforms we're discussing here (Coursera and Udemy), you have things like:

https://ocw.mit.edu/

https://onlineeducation.caltech.edu/courses/certificate-gran...

Professionals who know there subject are still the best way

That's true, but they are also a) a lot more expensive and b) unlike LLMs, the vast majority of professionals have family and friends and need sleep and food, and as such are not available 24/7/365

Hallucinations has made huge progress over last 3 years

Yep, now there are way more sophisticated. Same amount, though.

Nah they have definitely reduced massively. I suspect that's just because as models get more powerful their answers are just more likely to be true rather than hallucinations.

I don't think anyone has found any new techniques to prevent them. But maybe we don't need that anyway if models just get so good that they naturally don't hallucinate much.

That's because they're harder to spot, not because there are less. In my field I still see the same amount. They're just not as egregious.

Not in my experience. For example models often say "no you can't do that" now whereas they used to always say you could do things and just hallucinate something if it was impossible. Of course sometimes they say you can't do things when you can (e.g. ChatGPT told me you can't execute commands at the top level of a Makefile a few months ago), but it's definitely less.

> They're just not as egregious.

Uhm yeah, that's what I'm saying. The hallucination situation has improved.

They haven't reduced one bit in my experience.

Reply on news.ycombinator.com