Hacker News Reader

Comment by Zambyte | Hacker News Reader

> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human.

I actually really disagree with this. I think it's easier to distrust things that are human like. We are used to distrust humans. Less so things that seem like neutral tools.

For example, people mindlessly fill out forms presented by large corporations asking for personal information. I think people would be less inclined to trust an LLM with that information, even if it it's actually likely to be less actionable (at least with current capabilities).

View on HN

This is a great idea for an experiment.

I’m not sure how it would go.

Some people would refuse to give info to both the simple form and the personality rich chat bot. Some would give info to both. But how big is the middle area?

One thing is certain though — whatever format is more successful at getting peoples info — that’s the one that you’d see more and more of over time.

I was reading The Hacker's Mind by Bruce Schneier and he relays an experiment where a clearly imperfect navigation robot gives wrong directions to people repeatedly. Then a simulated emergency occurs, and the robot changes to saying "follow me to the exit" basically and points to a dark room instead of the routes the people took just minutes before, and despite having experienced the robot doesn't navigate well, people tend to follow the robot's instructions.

Welp.

I’m not even slightly surprised, but i fact checked this anyway. Here’s the article — and it’s even more alarming when you read the details.

https://www.schneier.com/blog/archives/2016/04/people_trust_...

Thanks for the interesting example / terrifying insight into human nature.

This is tangential, but I really cannot understand why you would think data provided to LLMs would be "less actionable." Our own dialogues, questions, and interactions are creating the most accurate and informative personality mapping available. The amount of data harvesting available (and certainly happening) here dwarfs even Google, because Google [mostly] loses you after a click through while for an LLM they get to capture the entire interaction on any given topic.

Intelligence agencies, politicians, advertisers, and so on would pay a tremendous amount to be able to query this data. And OpenAI is being backed by Microsoft, while Anthropic is being backed by Google and Amazon. Companies all well known for their tremendous regard for user privacy and rights...

> We are used to distrust humans.

I think this assertion might be cultural. I don’t believe that I distrust humans by default, I think I’m actually predisposed to trusting them and wanting to help or cooperate with them.

What is your credit card number?

If you don't feel comfortable telling me that, would you instead take a calculator of your choosing, and find the cosine of your credit card number, and behold the mighty value it returns? Or maybe would you buy something online? :)

I think you would probably be fine with the latter but not the former. People often don't even realize when they are sharing information with tools (see: how much people carry their phone everywhere). But you would probably feel uncomfortable telling me, a random internet stranger, everywhere you have brought your phone with you in the last month, let alone ever.

We have a certain degree of trust for people and a certain degree of trust for tools. I think people almost strictly will trust the latter more than the former with sensitive information.

Ah, now you're asking me for a piece of info that we're all predisposed not to share =P. You're right, in that situation I think everyone's much likely to give a credit card number over to a tool with no personality and less likely to give it over to a human.

> We have a certain degree of trust for people and a certain degree of trust for tools. I think people almost strictly will trust the latter more than the former with sensitive information.

I agree with this, I just personally feel that I lean more toward trusting people. But honestly it all depends on the context of what information is being asked. Your credit card number is a great example.

For an example on the other end of the spectrum, I'd be a thousand times more likely to hand over my real email address to a person than I'd be to give it to a tool (i.e. web form, app, etc.). These days I only assume the worst intent when something asks for my email address, but with a real person I know they just want to talk to me.

In general I would agree.

But AI can take the all the positive traits from humans to sound as likeable and trustable as possible (and companies curretly do that).

Social engineering is a thing. We like and trust some people, and some we don’t, without any evidence about their behavior history.

And, doesn’t your example conflict your intial claim? Because people trust humans, they send forms.

In my example, I don't think people would give that information away as easily if not for the form mediating it. I don't think a language model would be as convincing as a simple form.

I say this from experience - even using a language model running on my own machine, it sets off my internal alarms when I tell it private information, even though I know literally no human besides me will ever see it.

Reply on news.ycombinator.com