'Sweetie', a computer-generated girl, lures sex predators | Hacker News Reader

'Sweetie', a computer-generated girl, lures sex predators

nzherald.co.nz

21 pointsdsego57 comments

Lawyers here in The Netherlands are saying that those sex predators cannot be prosecuted because no actual girl was harmed. Furthermore this could be seen as incitement, which lowers chances of prosecution even more.

So while this surely raises attention for the issue, I'm not at all sure if it's a lot more useful than that.

This is my concern, too: there was no crime outside of thoughtcrime, and I'd rather we didn't start prosecuting that. That these men intended to engage in child exploitation doesn't alter the fact that they didn't.

However, this system could be used to identify people who would be willing to commit these crimes, and to justify investigation into other past crimes of the type that they may have committed. The line will probably be blurred in practice, but at least there is an ethical way to use this technology.

You really think it's ethical to entice people into thinking they're committing a crime, just to have an excuse to investigate them? That seems kind of shady to me.

A man that searches the Internet for vulnerable children, and then makes contact with a fake vulnerable child with the intent of abusing that child, does not appear to have been enticed that much.

Probably analogous to leaving fake angel dust on the counter in the bathroom of a club, then arresting anybody who tried to snort it.

No it's not. It's analogous to posing as a dealer but only selling fake angel dust. Which seems to me like it's a proper way to do it.

Attempted crimes are charged and convicted all the time. It isn't the thinking that's the crime; it's the attempt. The fact that they failed doesn't quite negate the fact that they were sincerely trying to get a real girl to do it.

At least existing American legal standards define "thought crime" literally. You can't be convicted of a crime merely based on your intentions, but if you take a "substantial step" (or something similar, depending on the jurisdiction) towards committing the crime, then it's no longer just "thoughts" and you can be convicted for an attempt. At least some of these men completed every step that was under their control: engage the girl and pay the money. That's an attempt, not just a thought crime. It's like lining up and firing a shot at what turns out to be a mannequin cleverly dressed as your ex-wife. Still attempted murder.

How can you take substantial steps to commit a crime against someone that doesn't exist?

Coming next: NSA pre-crime division (powered by AI precogs)

In the U.S. the theory of attempts is largely based on the Model Penal Code (http://en.wikipedia.org/wiki/Model_Penal_Code) (MPC).

The MPC has a somewhat unintuitive but very rationally consistent theory of criminal guilt. The MPC focuses on personal culpability. It finds someone guilty of an attempt when: 1) with intent to commit a crime; 2) he proves himself willing to carry out that intent by taking a substantial step towards committing that crime. The focus is on subjective culpability: what the criminal is thinking and what actions he takes. The focus is not on what happens.

Thought experiment: three people go out and buy a gun, and take a shot through an apartment window at who they think is their ex-wife. First one hits and kills her. Second one misses because of a sudden gust of wind. Third one hits what turns out to be a mannequin. From the point of view of moral culpability: aren't the three people equally guilty? Each had the same murderous intent, and each took exactly the same actions to carry out that intent.

Now, if you think the three aren't equally guilty, because the same thing didn't happen in the three cases, consider this hypothetical: three men are driving down the road. They see their ex-wife at a bus-stop. The first man is driving very carefully, but drives over a nail, blows out his tire, and veers into his ex-wife, killing her. The second man is playing a video game on his iPad, and veers into his ex-wife, killing her. The third man is still bitter from the divorce, and veers into his ex-wife on purpose, killing her.

If the focus is on what happens rather than on what the actor thought and did, should all three be prosecuted as murder?

Those are good thought experiments that really bring clarity to the issues. Thanks for taking the time to lay it out.

Is it naive of me to say that you should weigh in both the outcomes and the intentions to determine the degree of punishment?

So in the first case, the guy who actually hit his target gets a more severe punishment than the other two.

In the second case, the guy who demonstrated clear malicious intent deserves a more severe punishment than the other two.

The law does weigh both.

It's not naive, it's in fact how attempts are punished in most real jurisdictions, which, unlike the MPC, punish attempts less than actual crimes.

What's the purpose of the law? Is it to give redress to victims and punish perpetrators, or is it to punish criminal intent -- that is, thought? Do we view criminal law as a way to enforce societal rules against harming others, or do we view it as a means of punishing people who harbour antisocial intentions?

I suspect the answer is "both", but I'm leery of using criminal law to enforce consensus views on how one should think. I'm far more comfortable with using the law to address actual harm than to punish the desire to cause harm or even an attempt to do so. I realize, however, that there are many cogent counter-examples: being convicted of drunk driving does not require you actually hurt someone, trying to buy drugs in a sting does not require there actually be drugs present, etc.

First, you have to keep separate three distinct things: 1) criminal intent; 2) actions to carry out that intent; 3) concrete injury to some victim.

"Thought crimes" refers to punishing people for (1) only. But just because (3) is lacking does not mean we're talking about "thought crimes" so long as (2) is present.

Second, the purpose of the criminal law is not to give redress to victims. That's the purpose of the civil law. The purpose of the criminal law is to punish those who are morally culpable and to create societal norms certain kinds of behavior. That is precisely why (3) is not necessary to prosecute a crime.

> The purpose of the criminal law is to punish those who are morally culpable and to create societal norms certain kinds of behavior.

The purpose of any law should never be to punish the culpable. It may very well be reasonable to punish the culpable if it results in normalizing societal norms by way of a threat deterrent ("if you do crime $x, you'll get locked up just like this bad boy here"), but simply punishing a criminal for the sake of punishing -- hurting, torturing, locking up -- is an idea that I don't think stands to any thoughtful, philosophically-informed scrutiny.

I think if we get over this type of thinking society will improve. Now and then whenever debate about capital punishment gets picked up, people point out something about the histories of serial killers: almost invariably they've had particularly shitty childhoods, they were born to drunkards and criminals themselves, they were abused -- sexually, physically, they grew up in cultural filth, they had virtually no access to opportunity. The thing is, thieves, sexual predators and other folk who do sometimes petty crime aren't greatly dissimilar in this respect, they've also been enraged to some point to lose a sense of control, to lose respect for themselves and others. So to reform society for lasting good, the idea of punishment -- that you should make someone suffer -- is totally wrong. Instead we should should have social programs that hook criminals up with psychologists and therapists, see what's wrong, what's bothering them, give them a dignified place to live, and let them heal and let them rejoin society when they're better.

After all, this strategy works and gives good results where it's been tried: http://www.theatlantic.com/international/archive/2013/09/why...

That's an unsettled question - a political one, in many ways. We certainly have some interest in deterring criminal acts, as Rayiner explains.

Also, intent and thought are not the exact same thing in legal terms. For example, I have certainly daydreamed about robbing a bank - not because I want to take anyone else's money particularly, but because it's amusing to imagine what sort of clever tricks you might employ to pull it off. I'm sure you've noticed there's a whole genre of movies and crime novels about likable criminals pulling off an elaborate heist, doesn't mean the film production people or the audience that enjoys the films actually want to take up a life of crime.

Now if I start making all the preparations to commit a robbery - enlisting co-conspirators, or acquiring weapons, or suborning employees of the bank, etc. I've obviously gone quite a way past simple daydreaming. Such acts are demonstrative of my intention. Picture yourself sitting in a car outside a bank, with a bag on your lap that contains a gun, a ransom note, and a mask. You'd obviously be in a very different frame of mind from someone who was simply musing on the general possibility of robbing a bank, and a prosecutor would certainly be justified in asking a jury to take that into account if you were on trial for attempt or conspiracy.

Isn't the purpose of the law to shape society into what we collectively think it should be like?

Under this reading, the mechanics of how you choose what to punish don't really matter, so long as the result is closer to what you want.

Granted, we might want to live in a society where thoughtcrime isn't a thing, and this might shape the laws we create, but collective processes can be considerably less rational than that.

IANAL, but I guess the point is that your ex-wife exists, and can be set as a potential victim if your attempt succeeded. If you half randomly fire at what could be people, but it appears it was a mannequin you shouldn't be charged with attempted murder as there is no specific potential victim (but I suppose there are other charges for dangerous behaviour)

1) They often go after someone who is staking multiple targets. 2) Undercover cops act as streetwalkers in many districts and this could act as a precedent

Nobody was harmed in the interaction with the virtual child, but it's a reasonable bet that the 1000 individuals they've identified already have also paid real children for similar services, and their identifying details could either provide sufficient probably cause to get warrants against them or serve as points of correlation if police seize the data on the chat servers.

The show "To Catch a Predator" usually claims that the predators they catch are convicted in the footnotes at the end of the show. None of these predators actually engaged in any acts -- rather, it is always the intent they are being charged/convicted for.

It's possible that these laws are different in the Netherlands, but even then, predators located in the US may not be protected.

Then I would simply employ it as a means to get the names of those who deserve further investigation. The further investigation is putting a live person on the other side and then nailing the pervs.

Articles like these always talk about how huge the child pornography business is, but the language always strikes me as similar to the scare stories from the 80s about satanist cults, etc.

I guess I could be wrong, I don't know if there are reliable sources for that kind of information

The reliable sources are the hundreds of thousands of images of child sexual abuse on the hard drives of people arrested and prosecuted for these offences. Checksums of these images are made and distributed (to help find other copies; to help filter these images) and some people attempt to identify the victim in order to arrest and prosecute the offender and to offer help to the victim.

Satanist cults were always wing-nuts and piss-poor media reporting. Unfortunately, there's some awful reporting about images of child sexual abuse, and the perpetrators of child abuse, and the victims of sexual abuse. (Demonising sexual abusers is not helpful.)

The article claimed it was a "Multi-billion dollar industry". I don't think there's any compelling evidence that this is true.

> I don't think there's any compelling evidence that this is true.

Have you looked for evidence, or is that a feeling that you have? I'm not snarking, just wondering.

I guess it depends how you count the money. There has been a decline in the use of credit cards to access images of child sexual abuse. Most images of child sexual abuse are free, or traded for other images. But commercial trading exists.

The best site for modern information is Europol https://www.europol.europa.eu/sites/default/files/efc_strate...

You have people hosting images, and people paying for access to those images, and people paying to have those images made. You have people travelling to rape children. You have adults selling children into the sex trade. I dunno, that feels like it could be a couple of billion dollars.

Some offenders have millions of images in their collections.[1] In 2005 there were about a million images online.[2] About 50,000 images are added yearly.[3] Sometimes children are raped "on demand", with the video being streamed by the rapist to some one who has ordered and paid for that rape to happen on video, and who is directing the abuse.[4] Parents sell their children into the trade of images of child sexual abuse.[5]

Wired have some pricing information. http://www.wired.co.uk/news/archive/2013-10/16/child-abuse-i...

> [previous para talks about decline in use of credit cards, and rise of free images of child sexual abuse] New material has also become a currency in itself, with value ascribed to the novelty of the images. So individual video clips might cost as little as $10 (£6.20) each, with subscriptions for $50 (£30) for three months. Although one video file of new material on demand can cost as much as $1,200 (£748).

[1] http://www.theguardian.com/lifeandstyle/2013/oct/05/sold-mum...

[2][3] http://ec.europa.eu/dgs/home-affairs/what-we-do/policies/org...

[4] https://www.europol.europa.eu/content/new-cybercrime-report-...

[5] http://www.theguardian.com/lifeandstyle/2013/oct/05/sold-mum...

One way to judge that is to ask yourself if you know of any instances, personally.

Personally, I don't know anyone in a satanic cult. But I can personally name people interested in child porn (e.g., they have been prosecuted for it). So the second seems credible to me whereas the first does not.

I know 3 people in satanic cult, one of them even did criminal things. But I never met a pedophile.

If you read indictments and court judgments from sting operations on online child porn, the statistics are pretty depressing. It's been a while since I read any of that and have no desire to do so right now but the DoJ maintains a page about their investigations: http://www.justice.gov/criminal/ceos/subjectareas/childporn....

Finally the the Turing Test has been passed! This does not seem as impressive as when imagined decades ago. Fooling perverts, even to the point of sending money, seems to be much, much easier than creating more generally useful artificial intelligence. But I guess it is a step forward...

http://en.wikipedia.org/wiki/Turing_test

EDIT: As pointed out below only the video feed was artificial. The chat was with a live human. Impressive but very different than the classic Turing Test.

I thought that this was just a virtual face for a human researcher who was conducting the actual conversation - at least that was how the demonstration in the article was done. If they were entirely virtual I'm not sure they'd need 10 weeks to accumulate sufficient hits.

You are correct it was just a virtual video feed. I misread the article.

I wonder how successful they could be with current AI technology. It seems that in this narrow situation and very "motivated" opponent it could be very successful.

Agreed, but for these purposes it's probably better to have it human-driven to avoid the possibility of entrapment (where the criminal perpetrator is actively encouraged to commit a crime, rather than passively presented with the opportunity to do so).

Having said that I'd say it's only 10 years before there's a trade in AI-driven virtual sex partners, which opens up a whole other box of ethical questions.

There's a Spanish initiative called Negobot that seems to rely on AI much more than Sweetie does — as its name suggests, it's a bot, not just a decoy. (Article in English: http://www.huffingtonpost.com/2013/07/11/negobot-virtual-lol... and paper: http://www.esi.uem.es/~jmgomez/papers/cisis12.pdf)

I am glad that they said the current laws are enough. It always seems like people use these issues to get more power. I want to say something about how that probably works for their cause but I can only imagine most people[1] 'thinking of the children' and adding as many laws as they can.

[1]the public that elects officials

This is the YouTube campaign page. Obviously NSFW in subject matter, though not explicit. https://www.youtube.com/sweetie

One good thing is that they've started with the perception that pedophiliac behavior is a mental rather than solely moral problem. It's quite interesting, if you can put up with the awful 'dramatic' music.

This is the future of law enforcement in this field, and possibly others. The marginal cost of setting up virtual honeypots is now so low that the obvious strategy is to simply swamp the market with them.

Brave new world. An online world full of agent provocateurs that detect all crimes before they are actually committed.

A grim as it sounds, I prefer to see online underage prostitution rather than real prostitution.

I don't know.

On one side, this behaviour is wrong and should stop. On the other, it let those young girl make money without the risk of getting sick, pregnant or abused. The worst that can happen is her getting scammed.

But don't get me wrong. Child abuse should NEVER be encouraged.

This is a nice counterpoint to the article yesterday about a pentest team using a fake woman to spearphish their way into a US government agency that was posted here yesterday. [0]

[0]: https://news.ycombinator.com/item?id=6664572

At a risk of being downvoted:

I am not sure "webcam sex" hurts anybody. The girl is not physically harmed in any way.

what about emotional/psychological damage? I would imagine that physical rape is also mostly an emotional/psychological thing.

disclaimer: I'm not a girl, I have not been raped, I have not known of anyone close to me who has been raped, nor am I a psychologist.

A child (who does not legally have a capacity to consent) 'performing' for strangers?

You're going on a bit of a limb to suggest there's no possibility for physical harm here.

Well, there's a strong possibility that kids are beaten or otherwise abused if they decline to participate in such activity, which is one reason to undermine the market and make it unprofitable. Also, you're overlooking the possibility of significant psychological damage. Being a stripper doesn't seem like a healthy occupation for a ten year old.

The quote from their director was "Our worst scenario is that the same thing will happen with this as has happened with child pornography that is now a multibillion dollar industry in the hands of criminal gangs."

Also there was an article about cam girls the other day showing some/many are poorly paid in studios in 3rd world countries.

Where there is demand, there is always someone there to make money – in this case it's clear that attempting to curtail demand will stop commercialisation of this sort of thing.

The article mentions that the "webcam sex" involves people paying to watch, via webcam, actual sex. Very different situation.

There are examples of people actually killing themselves after webcam sex. Some of those people are blackmailed and asked to pay money in order to prevent release if the images. Some people just have the images released anyway, and are bullied to death.

Right. But are those the group that selects to go into web cam or the group that is showing off to a significant other/on a lark ?

Both.

Some people are lured by fake contacts into sexual explicit photography or videography. They are then blackmailed into providing money.

http://www.bbc.co.uk/news/uk-20698106

> The pair often pretended to be someone the children already knew on social networking and instant messaging applications,

> They would trick victims into giving them their online passwords using a link, before threatening them into "engaging in sexual activities via webcam," Ceop said.

http://www.bbc.co.uk/news/uk-23141414

http://www.bbc.co.uk/news/uk-scotland-edinburgh-east-fife-23...

> It is thought Daniel Perry had become involved in a scam where victims are befriended and tricked into making explicit webcam recordings.

> These are then used for blackmail.

So, if you had a daughter, it would be ok if she made a little extra money on the side?

It's kind of strange that anyone would have to explain why this is a problem. Wonder what just happened to your Google, Facebook, and NSA social graph profile? :-)

(I imagine you mean "young daughter" - this is about the problem that children under the age of consent are involved, not about women in general.)

I was unaware the only way someone can be harmed is physically.

I never understood this kind of field work when officers propose using prostitution/drugs/you-name-it services. It looks like a crime promotion: person might hesitate and never do a bad thing in years or they might do, but officers do help in it.

It's almost like a prison for thoughtcrime. How do you know the crime would happen if the person weren't helped(sic!)

So the Perverted Justice/To Catch a Predator folks are spared from Baumol's Cost Disease? (i.e. the problem in which your costs increase as fast as general economic growth because you have to use a real human on a mission-critical path)

Reply on news.ycombinator.com