Hacker News Reader

Comment by baxtr | Hacker News Reader

> This is how I make sense of AI reasoning. LRMs, chains of thought, thinking tokens: It’s wishful mnemonics all the way down — a heady mix of shorthand and suspended disbelief, like Oprah-style “manifesting” (opens a new tab) with a computer science spin. This isn’t necessarily a dig; all novel research likely requires some version of this mindset just to get off the ground. It certainly doesn’t mean AI reasoning can’t or doesn’t work. But the “wishful” part seems to be as powerful as ever.

“We react to language in a way that is very anthropomorphizing. That’s just the way that we humans work,” Mitchell told me.

I can definitely confirm the last part. Every time I read the output of an LLM, I picture a person talking to me.

View on HN

I think sensible legislation might require that commercial AI providers discourage anthropomorphisation by avoiding personal pronouns from chatbot interfaces.

"Hey, customer service chatbot, can you help me get a refund for my order?"

BAD: "Sure thing, I'll be happy to help you with that, I just need your order details..."

GOOD: "Yes, this computer system can start the refund process. Please enter your order number."

> discourage anthropomorphisation by avoiding personal pronouns

The fault is in the naïve antropomorphizer not in that which writes "I" being in fact a "first person".

Irrelevant. Humans are frail and we shouldn't design systems with dark patterns that take advantage of human frailty.

> we shouldn't design systems with dark patterns that take advantage of human frailty

First of all: «humans are frail», so the paramount social and individual objective, goal, is to overcome frailty (in cognitive and psychological and judgement functions).

Secondly: you are using a bad argument putting even innocent and somehow proper implementations in the set of "exploitative dark patterns".

And importantly: we the adults must not be bound in the Procuste's bed of a world for the use of children. If people cry when e.g. somebody multiplies 123 and 321, they must be brought back to health, and we must be in the condition of making our free and legitimate multiplications.

That's a lot of fancy words to say, "I think I'm too smart to fall for the same tricks that work on other humans."

How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

> How many hours did you spend on your phone last month? How many fewer hours would you have spent if the apps you interact with weren't designed to maximize "engagement"?

?? Very strictly zero? You are assuming we would install engagement-relevant third party apps on our palm computers?!

I use my time exactly as I decide moment to moment.

> fancy

Eh?!

> too smart to fall for

Very ill framed. We are adults, we behave like adults, and we need a society of adults.

Millions of adults get hooked by gambling, social media, and AI that pretends to be a person.

We need a better standard than "@mdp2021 thinks he's too grown up for this."

The standard is "do grow up; build a society that facilitates growing up; and do not hinder adults with rules for children - that is very much not a society".

And your insulting attitude disservices the loser more than the normal people: you know, there are people who can count, there are people who won't lie, there are people who do not take drugs... There do exist adults.

You're arguing that we should be deliberately build evil and manipulative technology... why? To assert your own superior discernment?

Nobody has said that one should build «evil and manipulative technology». An interlocutor that says "I" is not «evil and manipulative technology».

People (normal people) are not manipulated by a generator that says "I". There is an I in that generator (it is the generator that outputs its output); and those who have weakminded reactions in front of innocent phenomena (e.g. those who would believe that "My name is Ishmael" declares the name of the book, that their instance of Moby Dick would want to be called Ishmael) cannot propose their fault as a norm for the rest.

Those who said that e.g. the colour red arouses them beyond self control cannot demand that we put red paint out of commerce: they have no "right" to be wierdly aroused by the colour red, and we are entitled to using red and every other normal thing that is so grave to people with unjustified problems. If people fainted when one said "pudding", they should seek for health, and they cannot demand that we stop saying "pudding".

Their mental illness is "their problem" (not something that should condition us), and a societal problem - in terms that people should be facilitated in becoming healthy.

I suspect you will not accept this (because you have been manipulated by the generator?) but there is no "I" in repeated inference calls to large language models. There is no coherent entity behind the text. Try switching backend APIs mid-session, flip from Claude to Gemini and then to ChatGPT and the output will happily burble onwards "Oops, you're right, I see my last answer contained an error..."

It's lying to users just as much as an email autoreply that pretended to be a human response is lying.

> but there is no "I" in repeated inference calls to large language models

We do not assume that. We do not assume any "I" as any "persistent flow" with the current architectures. We find it bewildering that you assume that people - not just /some/ people - would do that. We call it legitimate that a formulator may say "I". Your idea of "I" and its constraints is just yours.

> as much as an email autoreply that pretended to be a human response is lying

Nobody in their right mind confuses an autoreply with anything else that some billboard.

> because you have been manipulated by the generator?

Duude...

I must repeat: there exist lucid minded people, not just the delirious junkies you see.

The last thing we need is governments mandating software functionality.

Boy have I got bad news for you...

I agree that these concepts seem a little vague and hand-wavy, but this is a) no substitute and b) far vaguer and unsubstantiated.

Reply on news.ycombinator.com