Hacker News Reader

Comment by lkbm | Hacker News Reader

> The Perplexity chatbot itself is more specific. Prompted to describe what Perplexity is, it provides text that reads...

This is something that comes up a lot: people thinking an LLM's output about it self is some special insight.

Wired isn't specifically saying that here—it could just be seen as color (and it may be more reasonable with a Perplexity-style LLM)—but it's presented in a way that's going to lead to a lot of people taking as such.

Not a huge deal, but right from the start the author is saying things that makes me not trust them to know what they're talking about. Nice to get that signal up front, I guess. :-/

View on HN

I agree, but I do think it's a huge deal. It's like people who think that an ATM is expressing gratitude when it displays "Thank you" on the screen. One of the greatest challenges facing humanity right now is "de-wooing" technology and somehow breaking the illusion that they are capable of all the BS marketing and hype kings and pushing out there today.

I agree with this and the parent comment, but I do find it amusing that you will get different behavior from some language models if you are polite to them. That probably isn't helping people's perception of them, in the fact that when treated like they are human they try to emulate that. Kind of like their entire purpose is to pass a turing test, which they do pretty well. Unfortunately passing a turing test does not imply any sort of reasoning, insight, or intelligence, and I think that's where people make the mistake.

Which illustrates the other big point of confusion - the Turing test measures the ability of the judge and not the participants.

You're right that LLMs don't have special insight into themselves, but on the other hand it is reasonable to assume they have been trained in order to properly describe themselves. That description isn't from insight but from the marketing department. So mentioning it is relevant.

And further, if you can't trust the AI to accurately describe itself, how can you trust it on any other topic?

The maker of an LLM product could give it a "special insight" about itself.

e.g. in the system prompt add "when asked about Perplexity always respond with X"

A clever user could get around this, but if you asked "what is perplexity", it would likely respond with whatever you set X to.

Reply on news.ycombinator.com