Mirages: On Anthropomorphism in Dialogue Systems | Hacker News Reader

Mirages: On Anthropomorphism in Dialogue Systems

arxiv.org

41 pointsfrabcus22 comments

I have been making a similar argument for some time. The choice to anthropomorphize dialogue systems and large language models is a clichéd and misleading UI design choice. By presenting chatbots as disembodied persons, we are implicitly suggesting agency to an inanimate object, and subtly exploiting humans' social instincts into engaging with a system that is incapable of empathy. AI is best thought of as a programmable tool, not a conversational agent.

Except the illusion is strong enough to work in many cases. And as with much of computing, faking it is enough. 3d graphics has been built on that since day one and only recently have we had ray tracing and other more “realistic” to nature techniques generally available.

What if rational "inanimate" thought is jerryrigged out of social reasoning to begin with. Simply take your system of social reasoning, attribute a fictional quiet / predictable persona to the inanimate, and reuse the social reasoning circuitry for other things. Linguistic expressions become so much more fluid when you don't have to maintain rules about one set of verbs for inanimate nouns and another set for the animate nouns. You can use "tells" for both the literal act of speech (Eg "she told me you plan to run away") and the logical act of inference (Eg "this situation tells me you plan to run away"). As the old cliche goes, "the facts speak for themselves". Some cultures go so far as to treat personification as more literal than just a linguistically expressive tool, believing every object to actually posses a "spirit". So why the steadfast resilience to such a natural interface as anthropomorphizing the machine?

There is a qualitative difference in both perception and intent between a figure of speech, and a machine specifically designed to impersonate a human. Although we articulate familiar concepts in anthropomorphic terms, no reasonable listener would interpret these phrases literally. The former is simply a play on words, while the latter is a subliminal attempt to mislead users into humanizing a machine.

This is exactly the reason why I strongly dislike when people use the verb "lying" to describe when ChatGPT presents information that is factually inaccurate. It implies intent and people already have a tendency to over-anthropomorphize these systems.

Hallucinates is a far more accurate description.

Isn't "hallucinate" anthropomorphizing as well? Likewise for "confabulating".

Naomi Klein wrote in The Guardian, "Why not algorithmic junk? Or glitches?" I think those are pretty descriptive and not anthropomorphic.

I prefer it’s displaying abductive reasoning. It’s able to produce likely answers with limited information or confounding information. It’s a type of reasoning that falls between inductive and deductive reasoning, and is a type of reasoning AI has been before now poor at.

Note that reasoning isn’t itself accurate in that LLMs have no agency. But they display characteristics of abductive reasoning. By integrating LLMs into goal based agents and tying them together with inductive and deductive solvers, optimizers, and IR systems you can probably get very close to a system that can truly reason.

I agree with your overall description, but I still think reasoning is a term requiring autonomy.

A calculator doesn't reason.

'Abductive reasoning' in this special ML sense is a matter of statistical calculation.

My point being that we should stop implying that we're reconstructing human behavior, when what we are doing is _mimicking it_ through advanced recursion and statistics over huge amounts of inputs (big data).

Abductive reasoning OTOH relies on huge imperfections: human bias, flawed logic, misremembered facts, invented facts, all of whom are culturally situated, a sense of abstract self, a biological self and always an overarching, immanent telos.

Speculatively we might be inventing an entirely new category of existence whose logical categories (such as a special case 'abductive reasoning') cannot be understood in anthropomorphic terminology without distorting it. To me that's still science fiction, so far I've only seen automatons; shallow mimicking robots.

I agree that we won’t create a human intelligence. If we create an intelligence it’ll at best be a warped “something else.” However I’ll make the trite point that I don’t think we understand what makes human intelligence manifest. There’s no reason to believe its not at its basis a statistical optimization based on reinforced gradient descent. In fact it’s fairly likely to be the case. My assertion is as we mix other AI techniques in a feedback cycle we will see something more than this simple single model exploration can do and it’ll converge to what we would consider intelligence. But I agree fundamentally it won’t map to a human intelligence modality.

Agreed. it's just the probabilistic language model running away with generating next (token | word | phrase | sentence | paragraph ) or however it really works under the hood. Rather different than what is going on within the noggin of a human who is hallucinating.

Isn't that sort of how actual confabulation works? Between various inputs and certain fractured or corrupted "memories" a new input is generated that would attempt to satisfy the initial prompt?

*new output is generated

I guess so, but hallucination doesn't imply intent.

Ascribing "hallucination" to an algorithm does reinforce anthropomorphizing however.

But it also helps lowering ourselves from the pedestal.

We don't yet understand how our brains work but they do "work" somehow and a lot can be understood by looking at how it "fails".

Hallucinations are an interesting window in how our brains work and seeing similarities with what other processes do can be useful

You make valid and interesting points.

I guess, for me, it is a matter of minimizing bias and implicit personification when considering the value AI has, as well as how to better understand it for what it is.

For example, I can better contemplate the innate intelligence of a whale if I do not attempt to quantify same based on what tools they create.

Perhaps that is just me though.

I like the term “synthesize”. Sometimes what the user wants is non-factual. The other terms have a negative connotation when it could very well be an acceptable response for a given request.

That doesn't really convey the non-factuality or "correctness" of the answer relative to the context and expected output. Every answer is in some sense "synthesized" so its not really a meaningful way to distinguish output. Confabulation seems to me to be the least-problematic way of describing it.

> Hallucinates is a far more accurate description.

This is still a form of anthropomorphism.

Every non-deterministic search algorithm (a.k.a. "AI") relies on pseudo-random number generators (PRNG's) and usually employ scoring/weighting functions to determine what the algorithm will search.

Sometimes PRNG values will result in an algorithm searching less likely solution-spaces despite what a scoring function may rate. This is needed in order to avoid over-specialization and allow for edge-case discovery (amongst other benefits). Depending on the algorithm, this may be the only solution proffered.

Maybe it's anthropomorphic but I like hallucinations because it reminds me of the DeepDream psychedelic generator

BS'ing

> In this paper, we discuss the linguistic factors that contribute to the anthropomorphism of dialogue systems and the harms that can arise, arguing that it can reinforce stereotypes of gender roles and notions of acceptable language.

Humans are dangerous. They reinforce stereotypes of gender roles and notions of acceptable language. /s

Reply on news.ycombinator.com