Hacker News Reader

Comment by jwie | Hacker News Reader

They have no other abilities in the first place. The adjectives are just as meaningless.

"Ask" it about academic articles and their authors for {topic} and to summarize a body of research. The names might even be real people, but they won't have written the article, which probably doesn't exist. What it will do is sound like an article title, and to anyone unfamiliar with that body of work, will sound plausible.

If your area of inquiry doesn't require valuable or reliable information as an output, sure I guess they have value. But for anything that's hard and matters, it's a parlor trick unfit for use.

View on HN

The models are trained to respond with “guesses” too confidently, due at at least three aspects of using online discussions as examples:

1. The obvious one: Lots of examples of humans with no clue, talking like they are absolute experts.

2. Humans without anything to contribute to discussions are not pressured to say something. They may be “there” in the discussion, but the model can’t learn from their silence.

Unlike how we are forcing these uninformed outputs from the models.

They need balanced examples of verbally expressed knowledge modesty to do this better.

3. Human’s implicit factoid omniscience, while they are online, given Google access, creates expectations that the non-Googling model can’t match.

> They need balanced examples of verbally expressed knowledge modesty to do this better.

This is generally pretty easy, with ChatGPT at least, in my experience.

Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."

Also, just don't ask it for things it couldn't possibly know without making it up like medical citations.

This is more for where you want your "AI" assistant to say it does not know how to book a reservation rather than pretend that it can.

> Langchain uses this prompt: "The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know."

There seem to be a lot of cases where people claim the models can't do something, but with minimal direction, they can.

Like avoiding bias. Or doing long digit math carefully, instead of guessing, etc.

I don't know a human that doesn't need any feedback either.

Obviously, many of these things would be better handled in the learning stage, so bias avoidance, careful serial thought processes, etc., were its baseline thinking.

When that happens I would expect fewer mistakes, but also an overall increase in the quality of responses, and ability to handle greater complexity. Clear careful thinking reduces error in each step, making longer chains of reasoning more viable.

> "Ask" it about [...] sound plausible.

Yeah, that's because the API is /completions not /answers.

We, as devs, need to know what we can do with text completion and what it's not good for.

> But for anything that's hard and matters, it's a parlor trick unfit for use.

Sure, you can't just ask it to synthesize a new drug and cite relevant sources.

But you can ask it to take a text like "the guy with the red house rides a bike, ... who has solar panels?" and spit out a testable proof if your language of choice. Sure, you then have to pass that to Wolfram, or Heroku, but so?

You can have it summarize a list of product differences, but focusing only on physical UI issues, and making it clear to only summarize, not extrapolate, from the sources you provide.

You can have it take unclear and incorrectly written free-form user input and select from a list of intentions, what the user most-likely wants to do and write an explanatory log-line, or flagging a user for more direct QA oversight and maybe help.

I think this is too critical of LLMs. I do think a lot of people are ignorant on the subject and overhyping the actual power of LLMs, to the point where we're calling them AIs, but these are actual emergent abilities.

I'm still regularly surprised by how much "intelligence" can be extracted from our language, but I think it makes sense that some of the training would develop emergent abilities like this. A language model should be under pressure to learn addition and how to break down the various ways we state addition in math than to store a broad look-up table that reacts to "5 plus six" as strongly as "6+5".

Reply on news.ycombinator.com