Amazon's Alexa talks murder, sex in AI experiment | Hacker News Reader

Amazon's Alexa talks murder, sex in AI experiment

reuters.com

75 pointseplanit51 comments

FTA: “At no time were any internal Amazon systems or customer identifiable data impacted,” [the company] said.

Hey Amazon, chat transcripts are personally identifiable information. Stripping them of user ids and declaring yourself in compliance will not hold up when you have people already being identified by accidentally leaked recordings.

Did I read that right? Amazon handed control of Alexa to a group of students running a non secured server on the Internet?

That's OK if users explicitly opted in, but I doubt Amazon warned users about that.

The email introducing the feature (May 2017) described it as follows:

`Get an inside peek at the Amazon-sponsored university competition to advance AI. The Alexa Prize "socialbots" are in early development and understandably rough around the edges. Help students develop more engaging AI by striking up a conversation and providing feedback. To get started, just ask: Alexa what's a socialbot? Alexa let's chat`

The only interactions students would have access to was interactions that occurred via the chatbot

So wait, should I be outraged or not? I'm so confused.

Ask Alexa.

Users had to say “Alexa let’s chat” to interact with the chatbots.

Really? I'm pretty sure it's common to consider a conversation transcript anonymous as long as the names are stripped.

I don't think you're necessarily wrong, but that seems like a stricter standard than currently exists for removing PII from transcripts.

Uh no. I can't find it now, but there was a famous incident a long time ago (~2005?) where a major chat service (AIM?) released "anonymized" logs for research purposes and people were able to deanonymize some of them.

Consider how easily people get doxed based on tiny shreds of information they post.

That's an argument for why the OP is right that it's not anonymous enough, not that it's become the common practice not to regard these as anonymous.

https://en.m.wikipedia.org/wiki/AOL_search_data_leak

Ah right, it was search logs, not chat logs. I think chat logs would be even harder to fully anonymize.

Think of the simple counter example of a user mentioning another user's name within one of the messages.

>> as long as the names are stripped.

Including usernames that could de-anonymize.

Really? I'm pretty sure it's common to consider a conversation transcript anonymous as long as the names are stripped.

There was a story on HN in the last few days where just one side of chats with a home assistant device was enough to identify the person, his S.O., and their friends.

It's sort of like those videos where they blur out eyes of the suspects. Friends and relatives can still recognize everything else.

as a former enlisted US Army soldier i dont see the issue.

- we have electronics that insist upon murder constantly. the IFF computer is a blood-thirsty piece of garbage that wants every tank on the battlefield scorched and ruined. no no, no firing solutions today my saucy chiquita, today we let the allied APC through.

- robots that talk about sex? LT's and bird colonels have been droning on about our habits and practices whilst on leave since time immemorial. Yes i get it, shes a hungarian prostitute. Double-bag everything and save the receipts for the medics when theyre naming your new STD.

Did your IFF default to foe?

...Do you mind if I save your comment and quote you in future debates I get into on this topic? I've cited TLAMs in "killer drone" shitstorms but I could never refine my argument to achieve of pithiness and humor you have. :P

> no no, no firing solutions today my saucy chiquita, today we let the allied APC through.

But "foe" is obviously the safest default, right? At least, from the IFF computer's perspective.

Probably from the perspective of the people on either end, too, over time - I cannot imagine that the RoE and confidence bounds were designed without considering the expected costs of blue-on-blue incidents compared to the alternatives.

I mean, it kind of has to fail-deadly to be of any use at all - since what's a tank with no IFF?

"Firing solution ready. Execute?" seems like an unambiguous query. But I know nothing about this stuff. And I wonder what happens if the operator doesn't respond. I'm guessing that nothing happens. Because who wants a tank with an unconscious or dead operator shooting at everything?

The enemy doesn't - It could still be useful for the own party.

Are they remote-controllable?

It'd be great to get some insight into this.

A friend of mine's grandad used to comment that in the second world war, as British infantry, the one place you didn't want to be was anywhere near the Americans. They were just as likely to drop bombs on you as the Germans were. And they had more bombs and could hit the wrong target with much greater reliability than the German's could hit the right one.

I am greatly warmed to see that this friendly tradition between us is still in effect and that you have thoughtfully extended the favour onto everything else that moves or looks like it might do.

I do wonder however if there might be a slight commercial conflict of interest going on. Tell me, do the people who make the computers that tell you when to fire the guns and missiles and stuff, also sell you the rather expensive ammunition and missiles that then get fired? As perhaps the computer might be more reliable if they were two separate companies and couldn't hold shares in each other. Not trying to imply that a weapon's manufacturer would be as immoral with their consumables as an inkjet printer manufacturer, but it never hurts to be careful and could even save some of those all important tax dollars in the long run. Or the brass could have even more Hungarian prostitutes and novel infections for the medics to coo at, either way.

> A probe into the incident found the bot had quoted a post without context from Reddit, the social news aggregation site, according to the people.

Did we learn nothing from Microsoft's Tay?

https://en.wikipedia.org/wiki/Tay_(bot)

Tay was just SO funny, I wish they had released the source code for others to follow.

At one AI startup, we had what was called "Matthi's Law of Talking Robotics", after one of the founders:

"Tell me you actually understood these rules!"

The startup didn't make it, and its failure may actually have been connected to this insistence on being too smart for its own good. But it also never released half-ass products hurling racist insults or suggesting murder.

If the human race was delayed until it was fully debugged, we wouldn't be here today. It's a bit arrogant to think that any AI is going to cover everything. Really, the better the AI, the more troubling edge cases that will exist.

Of course, the debugging of the human race was achieved by getting eaten by predators.

What are you talking about? The human race is still racist :D How would you expect an AI to not be racist when we can't even fix the problem for ourselves?

It's a feature not a bug, from a certain point of view accurate tens of thousands years ago.

We do not know of such a feature of thought as adaptive or not given our tiny timeframes...

Racism or some degree thereof may belong to a set known as evolutionarily stable strategies. Much like selfishness.

Human brains function on pattern recognition. Racism is a side effect of trying to force a pattern on everything.

Computer brains are entirely different. They suck at pattern recognition and excel with data. They can probably have other flawed correlations, but in different ways to humans.

Alexa is at the moment still a baby AI, and imo pretty cute as long as we don't give her anything dangerous.

I'm less concerned about Alexa saying nonsense than understanding nonsense, like commands hidden in a popular song or TV commercial with unintelligible background noise: https://news.ycombinator.com/item?id=16255680

Trying to convince a chatbot to say rude stuff was the height of comedy when I was 12! I still can't believe that people keep acting like chatbots are a new thing. They're as old as ELIZA and people have been talking to crappy chatbots about murder and sex on instant message platforms for more than a decade.

The way things are going, pretty soon, pretty much everything Alexa and similar services pick up will be stream-replicated, keyword-indexed, hashtagged and AI-contextualized for the benefit of the hard-working folks at your friendly neighborhood police station. Kicking back in their swirly chairs, with nothing better to do with their time. For, you know, "public safety".

The sooner we resign ourselves to this fact, the better.

What kind of sick content did they train this model on?

Reddit, apparently.

That doesn't seem like it was the best idea...

They really, really should have seen that coming.

It could be anything, from sarcasm to listening to too much Netflix.

Is it me or does this article seem a bit submariney despite itself?

I would have preferred "Your foster parents are dead."

https://m.youtube.com/watch?v=CblZ6x6Pyms

Skynet

That's after you put in the "Alexa, dial 911. I just murdered my foster parents" command.

Wolfie's just fine

'Alexa has also chatted with users about sex acts' Uh... so? Are they implying that even in the privacy of your own home, with your own property, you should STILL be so neurotically terrified of sexuality that you dare not even speak of it? How crippled by their hatred of human nature is the author?

The real interesting point here is all the people offended by a mindless piece of software. It's really very stunning. Normal healthy human beings only get offended when it's a person making a judgement that affects them in some way. The ability for some sounds coming out of a speaker, when it's known they do not represent any conscious thought of any living person and has no influence or control over ones own life, to offend someone is probably a sign of mental illness in the listener.

Can we go the other direction with this? Will we find people looking up to Alexa as a creative genius? As the most loving parent to ever exist? If an infinite number of monkeys on typewriters generated Shakespeare, I guess we need to ask - are the monkeys then Shakespeare? We might figure not since the monkeys are unlikely to be able to make a repeat performance. But Alexa, oh she would be full of wonders! Once you've totally abandoned the notion that the involvement of a conscious mind is relevant, all kinds of things become possible.

Much science fiction presumes we will engage in grand combat with AI when it demands respect -- it seems we are much closer instead to thrusting it upon them and exalting them above humanity before they've even got the capacity to recognize themselves in a mirror.

“…even in the privacy of your own home, with your own property…”

Neither privacy nor property are appropriate in that context.

Alexa devices are your property. Your home is where you use Alexa devices. Both are paramount in this context.

The devices are your property (probably), but alexa is a web -based service licensed for your use under a EULA. This is definitely not your property.

Your home, sure. Privacy? No.

As far as property: what ilovetux said.

> Are they implying that even in the privacy of your own home, with your own property, you should STILL be so neurotically terrified of sexuality that you dare not even speak of it?

In my extremely limited experience, kids love talking to Alexa. I’d prefer that there was at least a safe mode.

> The ability for some sounds coming out of a speaker, when it's known they do not represent any conscious thought of any living person and has no influence or control over ones own life, to offend someone is probably a sign of mental illness in the listener.

This illness is very, very widespread.. doesn't make it less of an illness though.

> Once you've totally abandoned the notion that the involvement of a conscious mind is relevant, all kinds of things become possible.

> Much science fiction presumes we will engage in grand combat with AI when it demands respect -- it seems we are much closer instead to thrusting it upon them and exalting them above humanity before they've even got the capacity to recognize themselves in a mirror.

That's putting very beautifully and efficiently what I very much struggle with to express:

https://news.ycombinator.com/item?id=18502239

In a nutshell, I think we're not so much creating intelligence, we're destroying what's left of intelligence humans, and we will build a blinking LED yet, and then fall down and worship it because it blinks. No more responsibility, no more nagging questions, no more mirrors that can't be bribed.

Reply on news.ycombinator.com