Hacker News Reader

Comment by simonw | Hacker News Reader

The most interesting thing about this post is how it reinforces how terrible the usability of LLMs still is today:

"I ask them to give me a source for an alleged quote, I click on the link, it returns a 404 error. I Google for the alleged quote, it doesn't exist. They reference a scientific publication, I look it up, it doesn't exist."

To experienced LLM users that's not surprising at all - providing citations, sources for quotes, useful URLs are all things that they are demonstrably terrible at.

But it's a computer! Telling people "this advanced computer system cannot reliably look up facts" goes against everything computers have been good at for the last 40+ years.

View on HN

One of the things that’s hard about these discussions is that behind them is an obscene amount of money and hype. She’s not responding to realists like you. She’s responding to the bulls. The people saying these tools will be able to run the world by the end of this year, maybe next.

And that’s honestly unfair to you since you do awesome realistic and level headed work with LLM.

But I think it’s important when having discussions to understand the context within which they are occurring.

Without the bulls she might very well be saying what you are in your last paragraph. But because of the bulls the conversation becomes this insane stratified nonsense.

Possibly a reaction to Bill Gates recent statements that it will begin replacing doctors and teachers. It's ridiculous to say LLMs are incredibly useful and valuable. It's highly dubious to think they can be trusted with actual critical tasks without careful supervision.

I think it's ridiculous to say LLMs are NOT "incredibly useful and valuable", but I 100% agree that it's "highly dubious to think they can be trusted with actual critical tasks without careful supervision".

Yeah that's actually what I meant to type

Capability today and next year will probably be very different in reliability

As someone who uses LLMs to write code every day, I don't see a huge progress since last year, so I'm also not that sure about next year.

“Replacing teachers” is of course laughable.

Teachers are there to observe and manage behavior, resolve conflict, identify psychological risks and get in front of fixing them, set and maintain a positive tone (“setting the weather”), lift pupils up to that tone, and to summarize, assess and report on progress.

They are also there to grind through papers, tests, lesson plans, reports, marking, and letter writing. All of that will get easier with machine assistance.

Teaching is one of the most human-nature centric jobs in the world and will be the last to go. If AI can help focus the role of teacher more on using expert people skills and less on drudgery it will hopefully even improve the prospects of teaching as a career, not eliminate it.

It's honestly so scary because Sam Altman and his ilk would gladly replace all teachers with LLM's right now, because it makes their lines go up, doesn't matter to them that would result in a generation of dumb people in like 10 years. Honestly would just create more LLM users for them to sell to so its a win win I guess, but it completely fucks up our world.

This isn't really a problem in tool-assisted LLMs.

Use google AI studio with search grounding. Provides correct links and citations every time. Other companies have similar search modes, but you have to enable those settings if you want good results.

Okay, but it's weird there is a "don't lie to me" button.

The "don't lie to me" button for a human is asking them, "where did you learn that fact?"

Grounding isn't very different from that.

How would that ever work? The only thing you can do is continue to refine high quality data sets to train on. The rate of hallucination only trends downwards on the high end models as they improve in various ways.

It's wikipedia in the 00s all over again being preached by roughly the same age and social demographic.

I’ve never had it give me bullshit citations when I specifically say look it up. It just gives me a clickable link.

This sort of concrete verifiable hallucinations can be trained out and probably will be soon

people have said this since chatGPT first released

Reply on news.ycombinator.com