Hacker News Reader

Comment by dreadlordbone | Hacker News Reader

I've never had it tell me it didn't know something, both GPT-3 and ChatGPT variations.

I think its a commonly accepted opinion that a person's ability to admit when they don't know something is a strength. It would be interesting to a see how much a constraint for known validity impacts the ability of a model to synthesize responses. In other words, where do we draw the line between knowing and "thinking we know" something (i.e., do we draw the line at 95% confidence)? Surely we do not hold each other to a strict requirement of 100% confidence, but there must be some acceptable range.

There's a lot of fan fiction out there in the Star Trek & Wars universes, maybe even some crossover stories. If the GPT training model includes that, then it might be fairly confident about an unfortunate incident on the Millennium Falcon involving Luke Skywalker and tribbles. Or does it understand the idea of "canon"?

That’s not a strength when it comes to AI assistants because it feels like an extremely common response pre-chatgpt

Just an example I found. It couldn't know the answer to this as it didn't happen AFAIK. It makes guesses for me all the time - it's usually when it's something really specific like a niche program on the cli.

> Why does darth vader make a reference about kittens in the movie

> Darth Vader makes a reference about kittens in the movie as a form of humor or irony. In the Star Wars franchise, Darth Vader is known for his serious demeanor and ruthless actions, so referring to something as cute and innocent as kittens is a stark contrast to his usual behavior. It adds a lighthearted moment to the movie and allows the audience to see a different side to the character.

I asked “What is Kurume coffee from Ethiopia?” and ChatGPT had a fine response - “I’m sorry, there doesn’t seem to be any information about a specific type of coffee called “Kurume” from Ethiopia… Could you please provide more context or clarify your question?”

It’s real though - I was drinking Kurume this morning, and googling finds a blog post from 2020 about it.

ChatGPT seems to only know it as Kudhum:

> Kudhum is a coffee variety grown in the Kaffa region of Ethiopia, which is known for being the birthplace of coffee. Kudhum coffee is known for its high quality, producing beans with a distinctive flavor profile that is characterized by a full body, medium to high acidity, and notes of fruit and spice. The variety is grown at high elevations, which contributes to its unique flavor profile. Kudhum coffee is considered one of the top-quality coffees from Ethiopia and is highly sought after by coffee lovers and roasters around the world.

Not knowing requires epistemic awareness, which LLM simply does not possess. I guess one can cleverly simulate it, but I doubt it’s the goal of OpenAI.

If you ask if for the weather in Paris, it will say it doesn't have that information.

It's trained to say it doesn't have information to recent data, such as today's weather.

But for questions that aren't based on recent data, it should still be able to say "i don't know" or "that question doesn't make sense" in the case of Luke Skywalker's problem with tribbles for example.

Edit: I do realize we're criticizing a computer program for not understanding the difference between Star Trek and Star Wars. Strange world we live in.

Reply on news.ycombinator.com