Hacker News Reader

Comment by ADeerAppeared | Hacker News Reader

There's a fundamental problem with all these "summary" tasks, and it's obvious from the disclaimer that's on all these AI products: "AI can be wrong, you must verify this".

A summary for which you must always read the un-summarized text is useless as a summary, this should be obvious to literally everyone, yet AI developers stick their heads in the sand about it because RAG lets them pretend AI is more useful than it actually is.

RAG is useless, just fucking let it go and have AI stay in it's lane.

View on HN

The solution is to feed both the text and the summary back to ChatGPT and ask it to identify any inconsistencies.

If you repeat these steps over and over for enough iterations eventually you will run out of money and the problem will be moot anyway

Fair enough, but what if I have access to infinite capital? Will I reach AGI?

You are asking an meaningless question: If I have everything, will I have this one thing

If you have everything, you have anything

If you have infinite capital, then you have AGI

No? They are asking if in the limit as amount of money spent in a particular way goes to infinity, whether AGI would thereby be achieved. They aren’t just asking “If I had infinite capital, would that be sufficient to achieve AGI by some method?”.

Your post is genuinely hilarious. Like perfect dry humour.

> The solution is to feed both the text and the summary back to ChatGPT and ask it to identify any inconsistencies.

Hmm... Sounds interesting, ...

> If you repeat these steps over and over for enough iterations

Oookay, continue...

> eventually you will run out of money and the problem will be moot anyway

Oh. :D

That's why you use a local model instead, that way you're out of money after buying the GPUs already and don't have to bother with the implementation taps temple

> There's a fundamental problem with all these "summary" tasks, and it's obvious from the disclaimer that's on all these AI products: "AI can be wrong, you must verify this".

> A summary for which you must always read the un-summarized text is useless as a summary, this should be obvious to literally everyone

Nah, it's still useful if the summary is usually right or mostly right. At the limit, it's not even clear that something can be summarized perfectly.

Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing.

Also, in my experience, these tools often fail when it comes to questions with a definitive answer. E.g. if you pass them a lot of text and ask a detailed question with a very clear answer, they often get it wrong. But when your question is vague like "summarize the text," they're very useful.

it's still useful if the summary is usually right or mostly right.

And you'd confirm that by having to read the unsummarized content. Thus useless.

Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing.

Reading an inaccurate summary is actually less useful than reading nothing. It's not like the utility of the summary is to exercise a reading muscle.

I agree. Heck, even reading clickbait headlines can cause adverse effects, even if they've summarized the text properly.

This is correct. There is a social context to this matter. The advocates of 'almost right is most cases is ok no harm done' are ignoring the (likely) operational and utility context of these tools.

You can just assume it's right or mostly right and move on with your life. We're not talking about designing spaceships here. Being wrong is allowed.

> Reading an inaccurate summary is actually less useful than reading nothing. It's not like the utility of the summary is to exercise a reading muscle.

No it's not. You're acting like these tools generately wildly inaccurate text. Even when they're wrong, they're mostly accurate. Almost everything people read is a mostly accurate summary, whether it's from some guy's article or wikipedia. We rarely go to the primary source.

edit - as a concrete example, I did my taxes this year with help from ChatGPT. It was a big improvement over using Google or reading through the instructions myself. And if it was wrong, well, maybe I'll get a bill or a check in the mail, but that was always a possibility and making a mistake on your taxes isn't illegal.

> making a mistake on your taxes isn't illegal.

It is when it amounts to tax evasion.

Tax evasion hinges upon intent. If it's a mistake, it's not illegal.

>> And if it was wrong, well, maybe I'll get a bill or a check in the mail, but that was always a possibility and making a mistake on your taxes isn't illegal.

If this is your approach to your taxes then you are probably wasting your time by using either of Google or ChatGPT. Just punch some numbers in to TurboTax based on a quick skim of the buttons, say “good enough for government work,” and wait for a check or a bill in the mail baby.

I don't want to pay someone for software to do my taxes.

But you want to pay someone for software that doesn't do your taxes but looks like it does? Or are you using someone else's ChatGPT subscription to do your taxes?

I use chatGPT for lots of things besides paying my taxes.

So it's not that you don't want to pay someone, it's that you get so little value out of accurate tax calculations that you only want to pay a tiny amount.

Just because you can amortize the cost of ChatGPT over more operations doesn't mean you aren't paying for it. You said you didn't want to pay someone for software to do your taxes, not that you wanted to pay only a little.

You're ignoring the case where I the author want to create a summary of my own work, to be published along side it.

It’s certainly not perfect, but I would definitely not class it as useless in many circumstances

Asking it to do a summary on a paper or meeting notes I’m mot familiar with, yes the utility is very limited exactly the way you’ve pointed out.

However, asking it to summarise or shorten a paper or meeting notes I’ve been directly involved in - that has enormous utility for me.

It provides me a quick starting point to create summaries, and I get to see if the salient points are addressed, and if not, I get to add them in.

I’m no a fast writer, and often experience writers block, so having a fast way to start is enourmously useful to me.

> Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing.

But that could be actually good alternative. Sometimes its simply not worth it. You can loose more time with the tool that will give you uncertain summary.

> Consider that the alternative to reading the summary often isn't reading the entire text yourself. It's reading nothing.

Having no information is always preferable to having wrong information.

Do you find yourself refusing to read articles or papers unless you’re first 100% sure that everything stated in them is absolutely true? Having to approach something with a certain amount of skepticism is not something that was newly introduced with LLM’s.

This is just a trust issue, which applies to pretty much any task where there is delegation.

If you ask an intern to summarize some text, you trust them to do a half decent job. You're not going to re-read the original text. The hiring process is meant to filter out bad interns.

If an intern gets it wrong, then you can sit down with them and teach them the correct process. Hopefully over time they get better, and once the trust is built you then stop being so involved. If they don't get any better, you find someone else to summarize for you.

You can't go through this process with an AI, every time it's just a shot in the dark.

With a specific model (just like with an intern), you only need to evaluate their work a certain amount of times to decide whether they're doing their job good enough to leave them alone and continue without further supervision

> If you ask an intern to summarize some text, you trust them to do a half decent job

Yes.

And if I were to propose that we filter all information we consume through completel unqualified interns summarizing it, I'd be laughed out of the room.

Yet that is the future all these AI firms seek to build.

What would be an acceptable error rate for your use case?

There are situations where AI is good enough, other cases where you need more accuracy, and others still where you should be reading the reference directly.

AI is improving quickly though, and context windows will allow for summaries to be tailored to each end user.

> RAG is useless, just fucking let it go and have AI stay in it's lane.

imho a side effect of promoting RAG is that the vector search by itself (on chunks of documents) might be a good-enough thing for most people. if we create a system without the LLM-summarization part, it might be the best of both worlds. Alas, people actually don't care about that stuff.

I am more curious why this disclaimer is missing on human backed products: "Human can be wrong, you must verify this". How do they overcome the fact that the information they provided can be wrong?

Agreed that RAG, especially undeclared RAG, needs to go. It's misleading and causes hallucinations for the majority of tasks.

What would you say to the standard counterargument that most existing processes that AI might aim to augment or replace _already_ have a non-zero error rate? For example if I had a secretary, his summaries _could_ be wrong. Doesn't mean he's not a useful employee!

The classic humans do it too fallacy.

Any specific rebuke in this case? Because it’s especially true for summarization, if someone doesn’t care enough to do a thorough job.

Yeah it's asinine if you think about it for more than a few seconds. The implication is that there is no nuance. Humans are imperfect and AI is imperfect so therefore they are equivalent.

If that secretary’s summaries were as consistently wrong and unhelpful as those ChatGPT generates, they would be fired.

No, they wouldn’t.

I regularly work with a wide variety of project managers, product owners, secretaries, etc…

I swear that most of them willfully misunderstand everything they’re told or sent in writing, invariably refusing to simply forward emails and instead insisting on rephrasing everything in terms they understand, also known as gibberish that only vaguely resembles English.

All of them are still “gainfully” employed.

Simple, that's a false equivalence argument that ignores not only error rates, but the quality of the errors made.

https://en.wikipedia.org/wiki/False_equivalence

The standard processes don't fail in the manner that AI does - they don't randomly start inventing things. Your paralegal might not give you great case law, but they won't invent case law out of thin air.

And if a paralegal did just invent case law, I'm betting they'd find themselves in a shitstorm of legal trouble.

Can human-generated summaries ever be wrong?

its*

Reply on news.ycombinator.com