Hacker News Reader

Comment by mertd | Hacker News Reader

I see trust as the big issue. Yes, it gives very reasonable answers. But how do I know it is not BSing, defrauding, or tricking me? You can ask for it to reveal its sources, but then you're back to Google style knowledge gathering of checking websites and judging credibility.

View on HN

There was a Twitter thread discussed here just a day or two ago where it was giving serious, in-depth, plausible answers to questions around Typescript.

Yet apparently those answers were quite wrong.

https://news.ycombinator.com/item?id=33817682

That trust factor is huge. I had this argument in the thread: when a human isn't certain of the right answer, they will (typically) provide answers, an explanation for their reasoning, and cautions about where they could be wrong.

An AI that sounds confident 100% of the time but is occasionally very wrong is going to be a real problem.

Why can't you hook it up to canonical references? If you asked it how to search those references then I feel certain it would be able to navigate to, parse and summarise the answers with ease.

> How many ways can you permute the letters in the word “safer”?

< There are 24 possible ways to permute the letters in the word "safer". This can be calculated using the formula for calculating the number of permutations of a set of items, which is n! (factorial n), where n is the number of items in the set. In this case, there are 5 letters in the word "safer", so the number of permutations would be 5! = 5 x 4 x 3 x 2 x 1 = 120. However, since the letters "a", "e", and "r" are repeated in the word, we need to divide the total number of permutations by the number of permutations of each of these repeated letters. In this case, there are 3 "a"s, 2 "e"s, and 2 "r"s, so the number of permutations would be 120 / (3! x 2! x 2!) = 24

Starts out just wrong, then explains the working and correct answer, but just completely loses it by the end. That calculation!

It seems to not calculate math formulas. Probably they turned that off because people would throw absurdly expensive operations at it

That's exactly how to tackle the problem. Using a search engine to ground it in real data. It's called a "retrieval augmented language model" and it basically means it first does a search and selects a few relevant passages, then solves the task using that additional information. The problem of verifying truthfulness is separate, but retrieval gives you grounding and references, both being necessary in order to use the model as a research assistant or teacher.

It's not a fundamental limitation, it will be standard issue soon enough - web enabled AI bot. It even has a flag called "Browsing: disabled" in its prompt, suggesting it will be enabled soon, probably after they find it safe enough.

How is this any better than what you'll get from Google?

Now that the cat's out of the bag, a lot of results may even be generated

Reply on news.ycombinator.com