Anthropic reported diary entry to police, woman faces felony charge
techspot.com
techspot.com
However, people need to get it in their heads that they're not chatting with their secret BFF, they're chatting with Big Tech. Before LLMs, Big Tech had no way to scrutinize the bulk of what was going on within their services, so you could have a secret hate diary in Google Docs. Now, everything you say or write can be automatically screened for red flags on a planetary scale, and probably will be because that's what the regulators and "concerned citizens" will demand. In a couple of years, you'll be biting your tongue a lot more often in private chats.
As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.
How much do you love Big Brother?
Most people don't realize that it is 99% like posting it on Facebook.
Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.
It's more efficient to have one central "verifier" for everything, but the "who watches the watchers"? question basically says: Either constrain by construction, have everyone verify (which are two sides of the same coin, btw, when looking at a "global" thing), or centralize explicitly.
I really hope you mean that in an insulting way to the industry and current legal situation, not as an actual solution.
It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
Custom negotiation is a big difficulty for individuals and the pricing will probably have fixed overhead that becomes pretty ridiculous when it's not spread over a hundred or thousand users.
> Or run a local model.
Much better advice. But still an obnoxious amount of effort and restriction to avoid spying.
> It seems like some people seem to think that a chatbot should guarantee total privacy like a confession to a Catholic priest or something. That is ludicrous and unrealistic. It's a commercial service subject to terms and conditions. If you don't like it then don't use it.
I want it to have the same privacy as using a local program. Does it still sound ludicrous if I put it that way? I'm not talking to a person that would have to suppress their own conversations later, and there's no need for any data from this chat to escape into the outside world. It's like wanting a phone line where the phone company won't listen in; even if doing that is standard procedure and I can walk away it's still gross.
In the case that the company running this program for me is served a warrant, I accept that saved data would be shared. But proactively screening my data for a reason that isn't to help me is bad. We should fight to make that easier to avoid.
Also you're talking out both sides of your mouth when you say it's "ludicrous and unrealistic" right after telling me I can negotiate that exact feature in an enterprise contract.
> But a civil agreement can't bind a counterparty to conceal evidence of a crime.
Are you talking about a different scenario? In a chatbot situation, the crime of "threat" requires snooping to even be possible. If the data is private there is no crime to begin with.
As articles like these show, cloud-based LLMs don't work in your interest today.
Have you ever thought of doing something you wouldn't be proud of, and then not actually acted on that thought?
If so, or if you can imagine a journal or always-listening assistant hearing your muttering thought. Should you be charged with a crime for that? Who is the victim and what are the damages?
Level 2, the slippery slope:
Suppose you travel to a country where your sexual orientation is illegal. If your AI company or social media provider knows your orientation, should they be compelled to reveal your crime? Why or why not?
This - specifically with a shooting, is basically the plot of The Drama (film) from earlier this year.
No AI in the movie plot though.
I had to move some games from C:\Games\RandomWarezGroup\GameName to my Steam directory, because Claude refused to reverse engineer an asset format due to piracy being incredibly immoral and illegal and blah blah.
Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.
On the other hand, they did put themselves into this position deliberately.
Show me any product, no matter how simple, that has no safety vs utility tradeoff.
For anyone technically inclined it should be obvious, but it isn't part of the zeitgeist or how they pitch it. People see it as being different than talking to a human, and behave as if there won't be a human in the mix.
(and not buried on paragraph 56 of a 20 page TOS that no one reads or understands)
Yes.
>Show me any product, no matter how simple, that has no safety vs utility tradeoff.
Show me a product that was not offered willingly.
Observe that Claude isn't one.
I feel like if people understood what these things actually are there'd be way less of this AI psychosis and similar stuff.
There'd also be fewer people falling for apocalyptic Rationalist delusions.
Also: people need to understand "not your computer, not your data." (Unless it's stored in the cloud but encrypted locally with keys only you possess.) Same goes for storing things unencrypted in OneDrive, Google Drive, etc. There is nothing to stop these companies from bulk scanning, data mining, or reporting people based on whatever request a government gives them. Don't count on them to resist, because they often can't, especially if the request is from a sovereign state where they do business.
Assuming you consider it a "delusion" to have a p(doom) of more than 5% or so, that's not uncommon among frontier lab employees who have a pretty good idea of how LLMs work.
There are people with education here that don't fare much better, so I don't know.
Um, this is coming from the researchers building these models. So, good luck?
Not applicable.
If the woman was chatting to a local LLM, there would be no way to charge her for a goddamn thought crime.
Which is what the government did here, with Anthropic's assistance.
Thinking about committing a crime isn't a crime.
As for treating the "threat" as "public".. something tells me if the court ordered to share something publicly, telling it to a chatbot would not count.
>Also: people need to understand "not your computer, not your data"
Rrrright, because it's so easy for people to know when stuff from their computer is transmitted to not their computer.
There's nothing physically stopping those companies from scanning data on your computer either.
And legally, surely the state that treats convo with a chatbot as public when it suits them would prosecute mega corps for overly thorough telemetry when it also suits them, right?
Why people here accept this as normal is beyond me.
A bot talking to you directly as if its some one real caters to your thoughts and can take you in a certain direction without you realizing it. I have heard first hand experience from people that they feel more comfortable talking to chatgpt or claude cause it gives a feeling of being on their side and listening to them.
Then the AI companies have more confidence that they can move forward in a certain way, and issue investor guidance that is maybe closer to reality.
They want stable rules they can follow, which will limit their liability as long as they stay within them.
They also would like those rules to be as restrictive as possible towards their competitors.
I'm quite sure that if there wasn't the existential threat of a lack of a moat, they would not be pushing for regulations at all.
It's really hard to take these companies seriously or at face value about anything they say.
I see constant ads on video platform (particularly youtube/tiktok) about llm chat apps, from friends, dating, romance and everythkng inbetween; that's personal.
People need to be reminded constantly if they use such apps that they are participating in easier mass surveillance, profiling and AI training.
It wasn't technically feasible to scan personal chats easily, other than grepping keywords which must have had a bajillion false positives. Now you can get everything autoscanned at scale.
For non-technical people, it isn't really news, because they already forgot about it after reading it. Maybe they'll be a little more monitored in their own typing for like... a day or two.
One of the hardest lessons to internalize, and keep internalized, as someone who works on and writes software, is the vast, vast, vast majority of the Public doesn't understand even the most basic shit about software. It just does stuff. Hopefully the stuff is good. That's it, beginning, middle, and end.
"Why would you think x would y" is a poor framing. They didn't think about x or y because they don't care. The phone works, that's the beginning and end of their interest in the subject.
I think in part it's selection bias? Like if you're smart enough to get by honestly, you're probably also smart enough to realize getting by honestly is just a way more comfortable way to live. The only reason you'd probably cross that line is because your principals, whatever they may be, conflict with those laws, or your life circumstances are so bad that you have no choice BUT to turn to crime.
And that cuts the other way too: if you're dumb enough to think you'd NEVER get caught for a burglary, for example, you'd probably be way more down to plan and execute one, failing to consider that most thieves aren't caught when they steal the shit, they're caught when they try and sell it later.
...but the last sentence if kind of important, right?
We only know about the criminals who got caught. Which probably means they did something dumb.
It's conceivable to me that there could be many—maybe even a majority—of criminals we don't know about because no one ever caught them. Maybe they committed fraud once, decided not to push their luck, and lived the rest of their life quietly. Maybe they killed someone and made it look like an accident, and no one ever suspected anything. How would we know?
Basically a lot more evil goes unpunished than most realize, because carma is a thing only when it's a thing - sometimes it just isn't.
Here's an FBI chart for 2019 [1]
Though some countries like Japan and Germany have really good clearance rates for homicide. [2]
But more heartening, it seems like it doesn't really matter how many people the cops catch. Crime rates are still generally on a long downward trend (with some bumps along the way). [3]
[1] https://ucr.fbi.gov/crime-in-the-u.s/2019/crime-in-the-u.s.-...
[2] https://en.wikipedia.org/wiki/Crime_clearance_rate
[3] https://ourworldindata.org/grapher/types-violent-crime-rate-...
I think it's also much about self-esteem -- am I proud of myself? And criminals probably think highly of themselves when they succeed in their crimes.
Also, unlike the bad old days when 1 in 3 was an informant, now ordinary people aren’t in “informant loop” of providing information on others, so they aren’t thinking about being informed on either.
Why should you be? Linux gives us software we can reasonably say we own, but not hardware. Everything you type into your local linux apps can be being monitored by your processor and whatever is sitting in the CSME/PSP subsystem which you don't have permission to access, but which is always running even when your computer is off. We need fully open, documented, auditable hardware if allowing any third party access to our devices means our private documents will make us all into criminal suspects.
On top of that, the data has to pass through several layers of routers and access points we control, which makes at least the fact of exfiltration visible. Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
It's possible for targeted shipments, but the prospect of it being done for all consumer hardware sold in the US is a bit far fetched.
Sure, I'll take your comment at face value, not all consumer hardware has spyware, but it cannot be ruled out now, in the future, or at least now for targeted high value customers.
Well, sure, there was NSA's Tailored Access Operations unit which interfered with all kinds of hardware, even computer monitor cables:
https://www.nbcnews.com/tech/tech-news/report-nsa-intercepts...
For these kinds of operations, open source hardware won't save you unless you build it yourself, much like open source software. At the end of the day, you're hoping that the pre built thing matches what it claims to be.
The hardware case of TAO is still targeted and didn't rely on the manufacturer's compliance and secrecy for their entire line of products, though. The scale of that to my mind is quite the qualitative difference.
Those “special workstations” are mostly about lack of features they view as potential attack vectors (external or internal motherboard ports, replaceable/non-soldered components, thunderbolt, anything extensible or “repairable”) and mandate certain hardware features (hardware tpm, locked secure boot, locked UEFI). Other than that, those workstations are Lenovo, Samsung, or Dell machines running typical chips. They may pick certain CPU models or chips but that’s mostly about avoiding new or “cool” features that may not have matured enough yet. Most of them are moving to thin clients though. Where you use that “special workstation” to remote into an VM that can access production systems. And that part is partially about controlling the software running in the VM and making sure updates can be forced “offline” on the VM even if you haven’t connected to it in few weeks.
We know for a fact the government does intercept hardware shipments already on some scale. It's a lot easier when there are only a few chip makers being used for most devices. Intel and AMD alone cover the vast majority of the CPU market for desktops and laptops. It's basically the same situation for the wireless chipsets in every mobile device. No need to worry about dealing with about every cell phone manufacturer directly when you only need to hit up the extremely small number of companies every manufacturer has to get their chips from.
> Plenty of people have a very strong vested interest in preventing data exfiltration from company hardware and so the government would have to somehow get in touch with every one of them to tell them to stay quiet before they leak any hints of it happening.
Whistleblowers are extraordinarily rare. Edward Snowden worked with many many others. Any of them could have come forward at any time, but they didn't. Only one AT&T employee came forward to tell the public about Room 641A (https://en.wikipedia.org/wiki/Room_641A). The government marched into AT&T's building, took over part of their offices, rerouted their network and the whole AT&T backbone into their offices and through hardware. You can bet that more than one person noticed that. Government has no problems at all with going to a business with guns and gag orders and telling people to keep their mouths shut. Companies have no problems keeping quiet about what's happening either.
No one who isn't directly in the know is going to notice if someone's CPU (which is running it's own network stack) sends a few extra encrypted packets going to the servers of major internet companies (microsoft, cloudflare, apple, google) to either be collected at those end points or just picked up as it passes across the internet backbone.
Maybe if we had a ton more competition in things like chip makers, ISPs, operating systems, etc. it might be more difficult, but the way things are it'd be easy.
You can have open and auditable hardware, but how can you be sure that the hardware you buy is exactly the same hardware as what's in the Github repo?
Wow, so we're assuming police state plus it's your fault if you didn't know...
I'm not talking about the lens of morality nor even the law, because it's obvious this shouldn't be the case but in actuality, it is - everyday more true than it was yesterday.
Of course, if I don't want anyone reading it, I sure as heck don't put it online.
You can bet that everything sent across the Internet is stored somewhere. But read? Yeah right. People don't seems to realize that there are 6B+ people online and just how big a number 6B is. If you have a staff of 30-40K people (like the FBI or NSA) and they spend 40 hours/week doing nothing but reading Internet posts and the average person spends 4 hours/week writing Internet posts (likely a huge underestimate) and the FBI agent can read 5x as fast as the person can post, then the collective manual surveillance capacity is about 2M people. That gives a 1 in 3000 chance that a random Internet user would be surveilled (though it is decidedly not random). With more realistic numbers of maybe 5000 analysts reviewing potential threats, spending 10 hours/week on it (so they have time to actually follow up on threats, attend meetings, communicate with their boss & coworkers, etc.), and the average person spending more like 25 hours/week posting on the Internet, that's a surveillance capacity of 10K people and your chances are more like 1 in 600K.
FWIW, I feel you; I too try to leave internet a mildly more amusing place to be.
Certainly it shows that agents like Muse are a complete nonstarter for anyone doing anything that needs to be private. Even if you never talk about it with the bot, merely having sensitive information on your hard drive means it can be sent to outside servers where, if it hits some safety filter, some probably low paid employee is going to read it. Say you’re working for a public company and have files on your computer that could be material nonpublic information, now potentially some content monitor is going to be seeing that and trading based on it. It’s completely untenable to have everything you put on your computer be subject to human review at some tech company.
I think you can count on governments accessing that information where it suits them, and you can definitely count on unscrupulous people like Altman within tech companies to do so too.
I’m not sure what the answer is here, but it’s naive to think that nobody is accessing the information you give away by sending it to cloud AI providers.
Anytime the strong encryption debate comes up it's actually about maintaining the effectiveness of unencrypted surveillance in the AI era.
That's definitely not the good word to use. It is most likely made with the intent to be privacy-friendly, but they are unfortunately anything but secure (including the whole Linux userspace), and especially in the age of agents it would be the best if everyone understood it that you are a single bash/npm install/malicious PDF away from everything bad happening with your data. But rest assured, your video driver won't get updated!
I'm also puzzled by the phenomenon of using ones real name on the Internet (outside a professional context). I believe this began to occur around the time facebook became popular.
no outside device comes in - no inside device goes out.
So I obviously opted to not enroll my phone, but that came to bite me in the ass one day when I needed to get a train ticket from my company email (it was a business trip) and the policy prevented non-enrolled devices from attaching or downloading attachments...
Felt like a fool grabbing my laptop to let the train attendant scan the QR code.
"Anthropic failed to report murderer's threats to authorities"
(or "Chatbot knew man was planning murder, yet company did nothing")
"Anthropic reported private chats to authorities"
(or "Arrested for chatbot fantasy")
To be fair to the journalists in these cases, there's also no society-wide agreed Schelling point about the correct outcome or correct rules. I have strong beliefs and intuitions about what should happen, but other people also have strong beliefs and intuitions, and many of those are probably opposite of mine. Even if my intuitions are the best and most justified, a journalist is unlikely to think "I'm just not going to mention that some people are mad at this company over this outcome, because a hypothetically better norm or principle would support the company's actions here". Hopefully the journalism can at least contextualize the lack of legal or social consensus and the difficult incentive problems, rather than jumping to "obviously companies are sociopaths staffed by supervillains".
Including any chats anywhere where someone might have a phone in their pocket, or if there's a "camera" attached to a utility pole or a nearby tree. The only real private chats might be whispered lying down in the bathtub together, with a mattress covering it like you're both hiding from a hurricane.
> they're chatting with Big Tech
They're chatting with any powerful person who wants to hear it. She thought she was chatting with Anthropic, who doesn't give a shit about her. But after being threatened (and immediately backing down because, of course, they don't give a shit about her) Anthropic has become an arm of the government. So she was chatting with the Bonita Springs, FL Sheriff's office, or anybody else. If I paid enough, Anthropic would tell me about what she was doing so I could sell her laundry detergent.
For Anthropic? They don't need it.
For the woman, whose entire crime was using a chatbot that runs on Anthropic's server, and wouldn't be charged or guilty of any crime if she used a similar chatbot on her own machine to write the same exact thing?
The woman whose only crime was using the cloud to store private information (diary entry, stream of consciousness, thought experiment, etc) assuming it will stay private, as is normally the case?
The woman who was jailed on a technicality that allows the government to treat the 1:1 conversation with a robot as a public threat because the fine print says that the corporate has the right to snoop on the conversation?
I hope the the woman gets some sympathy from whoever reads this news, because it's a travesty and perversion of justice that this took place and Anthropic assisted in.
This crap makes KGB look good.
It's the other way around, big techs need to properly disclose in their platform, during interaction that they aren't in a private and safe environment
OpenAI’s stated rationale was a concern for the shooter’s privacy, but its own interests
better explain its silence. Upon information and belief, OpenAI was seeking to avoid implementation of a
hard line rule to refer planning of real-world violence to authorities, perhaps due to how frequently its
product is implicated in threats to human life. Requiring such disclosures would be incompatible with the
company’s public position that ChatGPT is safe. It could also threaten the valuation underlying OpenAI’s
anticipated initial public offering. Rather than expose those risks, OpenAI accepted the consequences of
its silence. A mass murder in the only secondary school in Tumbler Ridge followed.
Accordingly, the Crown, led by Attorney General Sharma, and SD59 jointly bring this
action to hold OpenAI and Sam Altman accountable for designing a dangerous product, distributing it to
every home with internet access, ignoring the warnings of their own safety team,
refusing to notify authorities when they knew the shooter was planning gun violence, inviting the shooter back onto the
platform after deactivating the shooter’s account, and choosing corporate self-interest over the lives of
children. They seek compensation for the not just foreseeable but known harm OpenAI inflicted, the
damages that they incurred and are incurring, and injunctive relief to ensure that this tragedy does not
happen again.
That could be a meaningful difference
You can see it all in GP's link. It's very readable.
I figured it just didn't get flagged properly.
They draw the line using a team of humans who evaluate whether the threat is real and give a recommendation to management.
If you operate a gun store and someone reveals to you that they intend to use your product to shoot up a school you have a similar moral obligation, I think. This isn't a special case.
Considering that many people will share their deepest thought with LLM, it might start looking more and more like attempts of precog agents from Minority Report...
Google doesn't report people to the police for the private (non-CSAM) contents of searches or emails.
You could definitely go through people's shit before, but Big Tech generally had serious prohibitions and a stronger presumption of privacy about this sort of thing.
CSAM was an exception where they theoretically had the evidence of the actual crime, rather than writings alluding to them. And even that has been problematic.
Frankly, the charge here is total bullshit and should be thrown out. The law is meant to prevent people sending threats to others, not keeping notes. If this charge sticks, the law should be changed.
The problem with that is, "no, they don't have to do their thing. They have no given right to do it. If they do it, they should, they have an ethical duty to, do it right..."
Maybe there already is! But last I checked it was a little bit of a mess of manually installing things from multiple websites with little documentation.
If a person told their real BFF they were planning a shooting and went through with it, couldn’t the BFF be charged with accessory?
They did and they do
https://www.bbc.co.uk/news/technology-44699263
I never considered online services safe. What I object to is active scanning of content on devices. This should not be allowed.
Which it clearly wasn't, right? I mean, okay, in this case, the message did get reviewed by another person, but that's obviously an exceptional circumstance.
If I write something down on a piece of paper, and someone else goes through my garbage and finds it, is my note "communication made in a manner in which another person may view it"? It was clearly intended to be a private note!
Selling analogous profiles of companies using their services to the highest bidder would be another idea.
Whether intentionally or not, they market their product as suitable for all kinds of things that it clearly isn't when input is being used that way.
If we assume that "may" here means "could somehow" and not "is authorized to", the legal action after revelation seems correct as written (ignoring whether the person viewing it has any relation to the person being threatened) unless the law gets struck down as unconstitutional. The question is only whether Anthropic should or should not report it.
> "If I write ... It was clearly intended to be a private note!"
The law as written doesn't appear to distinguish about intent of privacy. Also, if you're in the habit of writing notes to yourself, I guess maybe don't write down the one that says you're going to shoot up the sheriff's office.
At some point I think these sorts of analogies break down, as the setup becomes too foreign to what we're more concretely used to.
In this case: there is no pen or paper which can store what you write on a replicated set of servers across the world, with an accompanying ToS telling you how that will be treated/used.
Although I have little sympathy for this women (both her intent and stupidity), I do agree this is a dangerous thing.
Today I learned about "foldering"[1], which uses the drafts folder of email systems as a "dead drop" between multiple participants. It goes back to at least 2005.
Back when I was in IT, circa 2010, I learned that several of the users of our systems routinely relied on the "deleted items" folder of Microsoft Outlook as a filing system, with multiple gigabytes important files stored there. 8(
Our customs are routinely ignored by everyone else.
The Spy's Son: The True Story of the Highest-Ranking CIA Officer Ever Convicted of Espionage and the Son He Trained to Spy for Russia
Then these people didn't change their ways when they were given adequate space.
Former head of the CIA, David Petreaus, was using the drafts folder of a shared Gmail account to communicate with his mistress.
LLMs seem to be moving toward personhood.
And after all the LLM learns from user posts too, and if everybody starts posting their darkest desires, fantasies, plans it may skew the "alignment" to say the least. Lets hope that the AGI level doesn't necessarily come with cheating, lying, religious fervor, power lust or whatever other sideeffects have been observed in human intelligence.
https://icap.law.georgetown.edu/wp-content/uploads/2026/02/T...
I am not a lawyer, but I find it hard to believe this statement meets that bar.
I don't know why people are even discussing this case like these companies will do anything on behalf of a consumer or think ethically at all.
Bots mass read and file and report all prompts that get categorized a certain way. They store everything else regardless. This will never change. It will only be exploited more and more. That's how this type of technology is deployed and progresses. Look at any other parallel. Like cameras or microphones.
I don't think it's worth dedicating even 2% of someone's day to avoiding surveillance. But I do think it's sad how many people don't realize all the utility they are gaining is lost once surveilled. There are ways around majorly bad surveillance activities that don't cost much money or time. The 80-20 is completely worth it.
You need to be able to use these models for the real world and not for some imaginary world where everything is safe and nice and happy all the time, while at the same time intensely surveilled in the name of CYA and the latest panic about whether speech THAT ISN'T EVEN BETWEEN TWO PARTIES is considered "wrong".
I'm a free speech fan that acknowledges there are lots of boundaries of free speech (fraud, perjury, blackmail, defamation), but the one thing that all of the boundaries have in common is that a second party must be involved for them to make any sense at all.
Maybe the courts will uphold this, maybe they won't, but don't take the risk!
If a policeman notices the sentence above on my phone screen during a routine traffic stop, the response you want him to take is... nothing?
When you read something describing in detail a person's intent to do something very bad, in a place where they write things that they intend to do, and which in the past they have in fact consistently done, you don't attach any significance to that at all?
Are you posting threats on hacker news while you are driving? And the cop was close enough to see your username and what you wrote? Is that why you were pulled over?
'Probable cause' should involve a degree of certainty, because 'possible cause' would be altogether too loose of a standard. It's possible that you're intending to shootme and you just mentioned saulpw to throw other HN users off the scent. Possibilities are only limited by the assessor's imagination.
The law in this particular case, which seems to be intended for threats that you actually send to someone, is being interpreted broadly to apply to any "threat" that you transmit to a server. So in your hypothetical, the legality would depend on whether your notes are backed up to icloud or not.
In my country, no "overt act" is required, but both here and in the US a "conspiracy to commit" charge requires an agreement with a second party. This is indeed consistent with a very broad interpretation of "no thought crimes".
I agree, and it's nuts.
This feels like less of an issue with anthropic per say as it is a broad reading/misuse of the law's original intent.
The commenters here are cute little HNers who think they have found a loophole in the law. They are not the first ones innover their head.
Spoiler: the law is written in words, and those words aren't strictly executed like in a computer program, they are interpreted by actual humans who can see what you are trying to do and will stamp it out.
Anybody showing a cop their hacker news comments at a traffic stop should be arrested, for harassing the police
Part of me says that the solution is stop entering any personal data into any device and service you don't own, but I'm not sure if that's really what we want considering that there are zero private cell phones. Even desktops and laptops aren't 100% owned by you these days. The only thing you can really do is keep them offline 100% of the time so they can't spy on you, but that seems like a lot to demand.
This argument holds no water at all.
I'm not sure why you think my argument holds no water when there are clear legal precedents that speech is not protected in some cases where there is "imminent lawless action".
Depending on context saying "I'm going to rob the bank X tomorrow" might also count as a threat?
But, in either case, writing this in a private diary could not be incitement or a threat because you are not communicating with anybody except yourself.
However, those other respondents to your post seem to be accurately describing the current legal situation. I asked Gemini, and apparently "conspiring" to commit an offense requires an agreement with another person in both my country and the US, where an "overt act" is also required (that may not be incriminating by itself). I find this alarming. The fact that someone's private diary entry describing in detail a plot to kill me does not amount by itself to anything is... incredible to me.
fork a child and kill it - Google Search google.com/search?clie...067j0203j0i20i263j0i22
kill child and fork parent - Google S... google.com/search?clie...o...1...5.0j0171j35i39j
kill parent with fork - Google Search google.com/search?clie.....1... ..0171|35139|33116
kill parent without killing child - Goo... google.com/search?clie...4589.0j32j1.0....1......
kill child without killing grandchild -... google.com/search?clie...5.0j37. ....о...1...5..0j3
kill all children - Google Search google.com/search?q=ki...&hl=en-bg&client=safari
kill child with fork - Google Search cooale.com/search2o=ki &hl=en-ha& client=safari
The current situation is a weird one. Anthropic reported single party interactions (per the ToS and common sense), there's a statue about sending threats (as there clearly ought to be), then somehow the definition of the word "send" was tortured by the local police. If a crime has been committed here it's almost certainly an infraction by the local authority against the spirit of the law.
I'm not saying fake child porn should be allowed or not-allowed, just showing there exist possible exceptions and rationalizations for them even without two parties.
I understand moral panic, disgust, etc, but rationally speaking.
You end up just weighing up the difference in trust between a vendor and a friend against the level of disinterest that they might have in your affairs.
edit: ah, future crime cannot be protected.
Lawyers have a duty of care to the court and they will absolutely drop your ass if you try to make them keep quiet about your crimes.
(This is not to be confused with them representing you in defense. You can tell them about your crimes if the government is trying to get you for said crimes.)
Sadly they’re now twice that price, which is a shame because I really want a second one!
I have a heretic modified version of it too, for when I want to use it for security and so on. Quite interesting
That dynamic will fuel further borrowing until we get to some kind of equilibrium, or some kind of washout occurs where interest rates spike higher and/or the value of GPUs and GPU services start leveling off or even declining. We might get a test soon as the FED has started hiking.
A cheap second hand 10ish year old card like my radeon rx570 with 8GB of ram is plenty enough to run a small uncensored model with llama.cpp if all one wants is chitchatting with a clanker.
We are not talking about heavy coding use cases here.
Got flagged for attempting to have Codex write a quick script to basically dictionary attack my own infrastructure with a tiny dictionary file.
A bit silly, but it doesn’t take a whole lot.
You can get around this by hosting in a 3rd party data center, but now you have the same trust issue again, but with more steps.
It’s all overkill for most people anyways IMO. This lady was just using it as a personal journal. Basically a glorified ELIZA. That kind of thing can be done with really small models locally these days.
No this is not true — colos are very hands off.
You don't need to operate a mini AWS to self-host, and I don't know why people on HN want to make it sound like self-hosting if you don't have nation state security
That’s a pretty broad statement? Maybe a more correct one would be that some are hands off enough that some people might find them worth it.
And that really only addresses one problem. The other one is the coordination problem. You go in on a $30k GPU with 3 other people. So each person puts down 7.5k. But then one person wants to pull out, and so every remaining person needs to put in an extra $2.5k on top, and they’re questioning whether $10k for a chatbot is really worth it.
I don't understand the coordination problem – if you want to pull out you have to find someone else to buy your share?
And she's facing felony charges. Yeah, folks on hacker news can talk about technical solutions all day long, but that's not the fundamental problem. The fundamental problem is that we've allowed our digital infrastructure that includes everything from banks to schools to doctors to become adversarial to us. In some cases predatory.
It's effectively untrustworthy, like being charged a felony for walking across a bridge incorrectly. That's not a "I'll host a small model in my garage" problem. It's a we need the government to do its job problem.
I agree in general that we technical solutions to social/political problems are suboptimal. But technical solutions don’t require the political system to work, which is a very attractive property at a time when politics seems broken.
There is a counterpoint that retreating into technical solutions only cedes more ground and makes things worse, and for that I don’t have a good reply other than the fact that life is short.
If you tell a teacher, a therapist or a priest that you are going to kill somebody, in many states they do actually have to report that and can be held liable for the resulting crime if they do not.
Every time we have a mass shooting in the US it inevitably comes up that they wrote it down or told somebody and then the next logical question becomes “how could we have stopped a mass casualty event?”
'No Way to Prevent This,' Says Only Nation Where This Regularly Happens
The people who are accountable and responsible are delegating their authorities to policy and now the policies are just implemented by machine systems and there’s decreasing human intermediation.
The humans that are still inside the system increasingly have less control such that it’s it’s increasingly difficult to find anybody who is the actual customer service type representative who has the authority to make a meaningful impact.
A PC at home with a 3090 in it is more than enough if you want to talk to a chatbot about your day.
Wow. I never knew that my Frankenrouter, gaming PC, and handful of laptops, switches, and WiFi APs were a datacenter. The things you learn...
> This lady was just using it as a personal journal. ... That kind of thing can be done with really small models locally these days.
That kind of thing can be done with MS-DOS's 'EDIT.COM'. If you're nasty, it can be done with 'ed', The Standard UNIX Text Editor.
For many, AI chats are damn close to extensions of our minds: diaries. Those are supposed to be private.
Was what that woman wrote a credible threat? Was she blowing off steam? Without knowing her deeply, how can you tell?
Maintain your privacy. Prevent thought policing.
I think Anthropic did the right thing here; but the sheriff's office are probably demonstrating why she dislikes them. Writing a diary entry to a chatbot is clearly not how this law was intended to be used.
EDIT: Actually, on reflection, making this report to the people she was upset about was probably not the right call. If they'd sent it to the FBI, there'd be a much lower chance that someone felt the need to assert their "authority".
In a "three felonies a day" universe?
Would you use a lawyer knowing they are inclined to turn you into the police if you talk about committing a crime in the future?
The law is the law.
We should be happy about this as for once the AI companies did the right thing.
Does all writing now have to be scanned for thought crime?
To me, journaling your intent in a private journal, whether that's an Apple Journal/Note or a Moleskine in your drawer, feels qualitatively different in some way. But I'm not sure why.
So "I'm going to shoot up the police station" written in your own journal feels somewhat different than "I'm going to shoot up the police station" said to a system that might be able to _interpret_ or _act_ on what was said in some way. Did I just give AI a legal duty, or even a soul I didn't think it had before? I've written up about three or four "what about this, what about that" and deleted them all.
"If you give me six lines written by the hand of the most honest of men, I will find something in them which will hang him."
The appropriate question is whether I want to live in Florida. This is much more a Florida law problem than an AI company problem.
It would apply if you happened to use Office 365 to write your diary.
I don’t live in the US. But this kind of broad law is hard to implement without surveilling all users, and it has multiple side effects.
What happens if I use Claude or ChatGPT to research sensitive social topics? Would that be considered a social network interaction and used against me when I apply for a visa?
Many governments (especially in Latin America) copy what the US does, meaning that similar laws will be pushed sooner or later.
Anthropic/OpenAI is not obligated to report your use, as long as it's not violating some terribly written law (like the Florida law). The government won't know about it, so no, you will not be denied a visa.
> Many governments (especially in Latin America) copy what the US does, meaning that similar laws will be pushed sooner or later.
I honestly don't know, but I suspect most US states don't have such an overreaching law.
But that service is a process on MY computer, loaded into RAM, with an abliterated model.
I don't trust 3rd party networks from abusing any data I give them.
The 3rd Party Doctrine destroyed the 4th amendment and is the reason privacy respecting software has to play legal games. E2EE while a good security practice shouldn't be necessary to protect you against the cops rummaging around your stuff. The bar to establish that information is private shouldn't be "literally mathematically inaccessible but the cops are still allowed to try."
While the privacy around ai chatbots is rotten in general, I can’t fault anyone who reported this.
They shouldn’t be in a position to report it at all.
This is the paradoxical times we live in right now.
Don't do something? She walks into the office and start shooting the place up. Several officers and innocent people are killed. Cue the media claiming, "You should've known she was talking about this an AI bot! Why didn't the bot tell anybody she was planning a mass shooting?!"
Do something? She gets rolled up by the cops and questioned about what she was talking about and brought to the cop station and interviewed. Cue the media claiming, "This is an unethical way to use AI, this is an infringement on free speech! This is authoritarian!"
I believe in free speech as much as the next person. But in this day and age, its almost better to be safe than to have to explain to someone's loved ones you had to chance to prevent this and did nothing.
It's always been complicated like this. That's why certain professions (psych, lawyer, clergy) come with rules around when and if disclosure is allowed[ required, and/or admissible].
As a Brit, I'm aware of https://en.wikipedia.org/wiki/Twitter_joke_trial
I can't say I actually disagree with the initial prosecution. The penalty was a fine, likely less than the cost of investigating it.
Intended as a joke? Blowing off steam? I can understand that, but given the number of people on social media is large enough to include genuinely unhinged people, you can't expect anyone who receives such as message to take them as a joke.
Same with AI use. A billion users, you have to assume some of them are actually sincere if they write about any act of violence, from self-harm to a plan to steal a nuke and use it in a false-flag attack to trigger WW3 and everything between.
That's a distinction that matters to me. Sending a spicy note to an LLM isn't remotely the same thing as posting it on social media where the entire world can read it.
It absolutely will misclassify things, it doesn't know any better.
(We should all wish each other good luck, because we're going to need it).
So this conundrum is at least partly of their own making.
The companies making these AI chat bots are the people who most want to surveil all their customers. The plan is (or will end up being): spy on what their customers are saying to their chatbots, use the conversations to further train/improve the AI and to increase the user's engagement with it, mine the conversations for every scrap of private/personal data and build dossiers on the customers, let other companies and governments pay them for that data or at least turn the chatbot into a shill/manipulator targeting customers based on the contents of their dossier, then use the contents of their customers dossiers for anything else they want.
If they actually set up their services to be private it would kill almost all of the expected value and also cut them off from their best remaining source of training data that isn't the AI generated slop their products are increasingly polluting the internet and the rest of media with.
This quote is pretty old:
> Those who would give up essential liberty, to purchase a little temporary safety, deserve neither liberty nor safety.
> its almost better to be safe than to have to explain to someone's loved ones you had to chance to prevent this and did nothing.
An authoritarian government isn't safe. That's why safety is also not deserved when you go chasing for a little of it at the cost of essential freedoms.
Hopefully more of these stories push people towards local models :)
With the obvious IANAL, it doesn't seem to rely on the message be sent to the person being threatened. The specific segment is "in any manner in which it may be viewed by another person".
This may be one of those cases where we get to find out how courts view SaaS platforms.
According to Gemini, "Florida appellate courts have overturned juvenile convictions [based on this law] when the state could not prove the person subjectively intended for the record to be seen."
Note that the law doesn't forbid the writing of a threat. You have to send it to someone. Had she kept it in a book under her bed, she would not be in trouble. But she sent it to a website/service/LLM portal.
>> It is unlawful for any person to send, post, or transmit, or procure the sending, posting, or transmission of, a writing or other record, including an electronic record, in any manner in which it may be viewed by another person
FYI, the use of drafts folders to transmit messages has been used by terrorists. This is likely where CIA director David Petraeus got the idea when he needed a secure way to chat with his mistress.
https://www.findlaw.com/legalblogs/technologist/gen-petraeus...
This is a huge privacy problem that is only going to get worse.
Saving is not sending ie passive vs active act.
What if she put it in a locked box before shipping it to herself UPS, and she has the only key?
What if instead of UPS, she hired a moving company to move the locked box?
What if she wrote it electronically in diary.txt, but it was backed up to a cloud provider?
--
I'm guessing there's some sort of "reasonable expectation of privacy" for certain activities. We're going to find out what Florida courts think about this new medium.
Does the person have to know (or at least believe) that it will be viewed by another person?
She likely didn't think anyone would view it. Honestly, even as a career software developer I don't think it is unreasonable to think know would would see what she wrote to an AI. I assume most of what I write to an AI is not viewed by any other human, based simply on the quantity of messages sent back and forth to AIs, I would assume a vast majority are not read by another human.
What if she had written this into google docs, and she kept a diary there? That also crosses state lines, and is transmitted to another location.
You can argue from technicalities but they would need to prove intent.
The AI companies have clauses in their user agreements saying they can review content flagged as harmful. It’s not legally spying.
If you recall previous outrage about ChatGPT being used in cases of suicides or shootings, this is the result. Every time a crime was committed and the police found ChatGPT history about the crime, the media turned it into a frenzy. So the AI labs added safety filters to their consumer plans that detect threats of violence, escalate them to human review, and report to the police.
Spying is not the right analogy because the information was given to the police by a third party which had a EULA saying they would do this. A more analogous situation would be someone reading another person’s diary and then turning it into the police department. There might be some limitation in the law that makes the evidence inadmissible because it was not intended to be shared with anyone, but that’s a separate decision.
This is spying with extra steps couched in corporate speak.
Frustrations about Anthropic’s EULA are a separate matter.
Presumably, Anthropic did the spying and the reporting.
You argued that it is not spying, since the spying may have been made sufficiently explicit in the ToS/EULA.
This raises the question: Does announcing a spying operation mean that it is no longer spying? I've never heard that perspective before.
Well, kind of, yeah; the dictionary definition of spying requires secrecy and lack of consent.
> to secretly collect and report information about the activities of another country or organization[0]
The only real debate is whether or not having a clause tucked away in a EULA that few people read makes it a secret. If Anthropic had a big flashing red banner that said "FYI we automatically flag and review any conversations about illegal things!!" on the front page nobody would call it spying.
[0] https://dictionary.cambridge.org/dictionary/english/spying
A less central case would be when you clearly do know about the activity but you can't quite see the details, like with behavioral ad targeting or something. It feels pretty normal to me to call that spying even if it's disclosed to everyone and certainly happens to everyone, but it's also a less central example of the concept.
You can call it anything you like, but only the legal definitions matter for the legal case.
If you change the situation then yes you can in fact change our responses. The problem is you then are no longer talking about the original situation.
It also bears mentioning that providing a dictionary link to “spying” is pretty patronizing/passive aggressive. On par with sending a basic Wikipedia page. You didn’t even bother to post the definition you want to apply.
The initial comment instead questioned how someone could be accused of making a threat if they did not realize anyone would read their private content. You probably also can not insult someone with a statement you never expected anyone but you will ever read.
Eh. Both Superpowers knew that they were spying on each other all the time, and that was still considered to be spying. But feel free to replace the word "spying" with the phrase "clandestine largely-automated mass surveillance" if it makes you more comfortable.
> ...and lack of consent.
Given
* the fact that the contracts one is required to "agree" to in order to use most services are often novella-length or longer, and frequently include by reference other contracts of similar length
* that nearly all contracts like this have a clause where not only does the powerful party reserve the -very frequently-exercised- right to change the terms of the contract without any prior notice, but said party presumes that you automatically accept the rewritten contract and gives you no option to negotiate
I'd argue that the real situation on the ground -in the US, at least- is that "consumers" have consented to approximately zero of the contracts that -despite that lack of consent- legally bind them.
You don’t need to presume. Anthropic reported it.
“Spying” as a legal concept has a definition that does not apply here. You could say they were “spying” in the sense that they read someone’s input, but that’s literally what they said they were going to do in the agreement when the person signed up.
So I responded to the question about the case being thrown out for “spying” by trying to show that the word doesn’t apply in the legal sense. If you sign up for a service that says “Hey we’re going to monitor your chats and might report things to the authorities” and then they monitor your chats and report things to the authorities, you should not expect the case to be thrown out for “spying”.
"Anthropic" does not read messages, it's an abstract entity involving many humans and computers, so let's be specific wherever possible.
> and respond to it in some way.
The computer is supposed to respond in a specific way that doesn't involve humans. Any reading/actions by humans is entirely separate and not expected.
> If you sent an email to a colleague threatening violence, you would not be surprised to find out it was reported.
And if I didn't send it, I would be surprised.
They get friendly and loose-lipped with the bartender over the span of months. Eventually they let slip that they plan on killing their spouse for a life insurance payout. At first the bartender thinks they're joking, but it becomes evident that there's an actual plan being acted upon and someone's life is very likely in imminent danger.
Does the bartender have a responsibility to go to the police?
/s obviously
Calling something names doesn't invalidate it. It only invalidates what point you're trying to make.
Then, you can write anything in an EULA but it is not automatically legal either.
So don't run for office or anything like that. Someone, somewhere will have a contact that will get that.
Obviously, it's hard to judge exactly what was appropriate there because we're being asked to extrapolate from a two word quote about the customer intending to "shoot up" the sheriff's office. Consider the following two statements, which express quite different levels of intentionality.
I got a $200 ticket from a sheriff's deputy today for throwing away an apple core. I'm so mad. I'd like to shoot up their office!
Those sheriff's deputies have exhausted my last reservoir of patience. I'm going to shoot up the department. They'll be sorry when they're sprawled all over the floor bleeding out from saucer-sized shotgun slug wounds. I can't wait to hear the screaming and crying of their miserable families!!"
I'm guessing that the diary entry was a more casual expression similar to the first statement, or they police would have quoted more of the statement to emphasize the apparent severity of the risk but it's hard to say without reading the charging documents.
sandbox your ai.
Any failure to understand what it can access or what it has permission to see from the user's end is presumably not their problem. Regardless of what the user specifically asks of the tool.
this is exactly what I meant. I am presuming the danger is AI reacting to personal notes that it reads on your computer, like a diary, and you should not allow the tools to have access to those documents.
The prosecutors likely know this and expect it. But there's enough gray area here for them to make the argument, and it's hard to prove malicious prosecution, so they know they'll get away with it. It's just about sending a message to the public - they don't care whether a conviction sticks. Just politics.
Of course I did not mean any of that stuff, but how can you make sure a human reviewer knows you did not mean it while the llm does not know that you did not mean it.
I guess its a miracle I am not in jail yet.
Flagging people for anything said to an llm sounds wrong to me because an LLM is not a real person and while some people put in their internal thoughts, others just roleplay and the two are inseparable just from reading it.
Adding: - I typically ask questions in the I form, regardless for whom or why I ask for. - Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-
But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).
A Minority Report like this shouldn't cause a referral to the police.
Damned if you do, damned if you don't. Same as with social media platforms. That's because only a tiny tiny sliver is for true privacy when that means "bad things might happen" or bad people, such as your political enemies, may do stuff you don't want.
The story you're describing would have triggered outrage. And this story will trigger outrage. And generally, the people who will be outraged are different people and we don't have to treat those two outrage reactions as morally equivalent.
But I can't help but wonder if it isn't some socially apparent phenomenon stemming from Simpson's Reversal [1] as applied to group sentiment analysis / polling of opinions.
[0] https://knowyourmeme.com/memes/the-goomba-fallacy [1] https://en.wikipedia.org/wiki/Simpson's_paradox
In my opinion, if the company is allowed to see the data and train on it, then they are also responsible for reporting stuff like this. Without knowing the data licensing agreement the lady had with Anthropic, if she agreed to letting Anthropic see her data, then they should do stuff like this. If she didn't agree, then I wouldn't condemn Anthropic for failing to report an attack
Stories like this article have zero effect on normal people. They see a crime being prevented, which is good. You have to bring a story where a sympathetic character is getting the short end of the stick somehow.
You would expect them to enforce every possible legal standard, in every jurisdiction.
Why stop with criminal law? It should flag torts too, right?
Absurd.
Google also monitors your searches, it’s to be expected that an AI lab will know all your prompts, they aren’t providing a paid service for free out the good of their heart. lol.
Er. When you actually pay, at least Anthropic gives you an option to not use your prompts in their training data.
Similar thing is that I believe if motorcycles were invented today they would most certainly be banned from the road due to their safety properties.
And as we progress we will need to rationalize what it means for private companies, AI, to know everything about you and for the government to have a tap into that. I hope balance lands in favor of things like privacy preserving underpinning.
> Heller faces a charge of making a written threat of violence under Florida law. Florida Statute 836.10 makes it a second-degree felony to send, post, or transmit a written or electronic record threatening to kill or injure someone, carry out a mass shooting, or commit an act of terrorism.
She didn't threaten anything, she wrote down that she was going to do it. A "threat" is more than a mere statement, especially when written in what is described as a "diary".
> A Florida woman is facing felony charges after she used Claude as a diary and allegedly wrote that she planned to "shoot up" the Sheriff's office.
Obviously I don't want anyone to shoot up anything, but this seems like a weak case legally speaking.
I can certainly threaten you harm and send it to not-you and you're still clearly in danger even if it wasnt transmitted to you. So the question becomes did she transmit it to someone? Clearly yes she transmitted it to Anthropic. But she clearly intended to send it to Claude, an inanimate object.
> The communication must be made in a manner in which another person may view it.
Even 'transmitted' is too broad if you also consider iCloud backup to be a means.
The clause in the law is pointless, since if you do something that nobody else can see, you can never be punished. But many, many, many laws are written without regard to whether they make any sense.
What this means here is, of course, equally spongy, but it is interesting as there might be an argument here that she did not intend for it to be viewed by anyone as, regardless of what the T&C say, most people do not expect their "private" chat logs between them and a machine to be seen by anyone at all.
It's not any different than telling an automated phone voice tree system that you plan on killing someone and then being surprised that your words were later heard by a human. She absolutely told a company's computer. She sent the message.
The law may have been intended for more direct threats to a person as a means of intimidation, but that's a separate conversation.
I'm not really sure that this can be likened to a diary when it is called a "chat" but that's for the legal system to determine, not me sitting on my couch.
And yes, some laws are "strict liability", I don't think this one is.
IMO I do not think this is a grey area and it's legal to tell a LLM you want to kill someone. It's certainly not a "threat" like you might send to another person, though it may end up being evidence of conspiracy or premeditation. I suspect we would be well served to, after a few years of experience, put together some laws governing when LLM chats must be made available to authorities.
It is very interesting that the LLM responses to these lines - the context around what she is saying - is not in the article. I suspect, as is the case in many instances where LLMs are involved in violent planning, that the LLM was urging this behavior on. Basically entrapment - you are encouraged by a robot to become more violent and vindictive and then when you do you are handed over to police.
> Review is needed to enforce our Usage Policy... designated members of our Trust & Safety team may access this data on a need-to-know basis as a part of their evaluation process.
[0]: https://privacy.claude.com/en/articles/10458704-how-does-ant...
The critical part of a "threat" is that the perpetrator takes some intentional method to deliver it.
Reporting the danger is by itself a good deed. But there should be a better way of restricting firearms from the probably irresponsible lady than using inappropriate charges to punish the thoughtcrime, OR waiting for them to commit violence.
— via https://www.leg.state.fl.us/Statutes/index.cfm?App_mode=Disp...
The DA is serious about "in any matter."
> may be viewed by another person
Was it viewed by another person? Yes.
To me, that is the more interesting legal question. Does a LLM-based safety net that sends content to a human, when the original use case would not have sent it to a human, count as "may be viewed by another person". It certainly wasn't intended to be, and that isn't the norm. At the same time, because no security is perfect, we could say that any digital record, stored in any way "may be viewed by another person."
Something for the courts to sort out, of course.
If you enter my house I will k!ll you. <- is this a threat? noone knows. The interpretation of the word "threat" is unknown. Besides, is a conditional a statement? who knows. But sure, blah blah... "in any manner in which it may..." these words are putting me to sleep.
One unfortunate woman who happened to write the wrong thing in the wrong place is now having her life turned upside-down for perceived thought-crime.
To Anthropic, and all employees working there, your company's product and the result of your work is cruelty. You are enabling it and pushing it down everyone's throat. You can never again claim that you are the "ethical" AI company, for no such thing exists.
Over in Europe they want to read all ofd our private messages, yet these chatbots, pretending to be our friends, will snitch on us just for our thoughts.
It's getting pretty orwellian out there.
What would happen if it had scanned a file it didn’t have permission to look at and found this threat?
I honestly don’t know how I feel about this. On the one hand if you’re using claude as a diary you have no expectation of privacy and she was talking about committing a very serious crime.
This still makes me feel queasy though.
For the sake of even more argument, imagine if she was writing her thoughts with a pencil, on a good old fashioned paper diary, and she had a phone nearby and the phone took a picture of her diary, OCR'd the words, and reported it to the police?
Anthropic (in discussion with Pentagon) claimed mass surveillance is their red line.
Yet, they do automated mass surveillance of their users on behalf of police.
> “You have the right to remain silent. Everything you say, do, or generate on this device can and will be used against you… Would you like to create an account?”
Is this an invasion of their privacy (reporting to police)? Yes, but possibly warranted?
Should a social worker have contacted them rather than the police? Probably, if for no other reason than to ask if they were serious about harming someone.
Difficult questions, I'm still undecided on whether it's OK to always ignore someone's rants, even if it may be (or they think it may be) a private diary.
Actually charging them with a felony seems pretty quick to accuse. (Maybe I missed a hint about how long the investigation took before the felongy charge?)
Of course, that could change if there's evidence that she took action in pursuit of a goal, like buying ammunition or repeatedly driving around the entrance to her alleged target.
Edit: I thought it was figurative language, but I actually think it did that warning when I asked it about what types of defenses stadiums had against drone swarm attacks from terrorists and why none had ever occurred and then kept probing it's excuses with technical workarounds. It got very upset and said even questioning in an intellectual/academic capacity was grounds for terrorism charges. Sheesh. So many rules these days about what one can or can't think about even when it's purley a thought exercise and there is no intent to do anything.
A diary constitutes making a threat?
Oh boy the roleplaying part of LLM world is in for a bad time
If they were to offer total privacy, is it ok for the public to use chat to get advice on _how_ to commit a crime? Basically everyone agrees that crime-committing advice is inappropriate…but if it is not ok to get advice, that means there must be a portal for law enforcement to step in when that may have happened. Then the question becomes what is the line for when to report? In other words, the issue needs to be adjudicated.
But we don’t want OpenAI/Claude to have some $20/hour reviewer making decisions that are this high stakes…we need the courts to do the judicial work because they (1) have a public charter, (2) have meaningful expertise and specialization at interpreting the law and (3) we can hold them accountable.
> If they were to offer total privacy, is it ok for the public to use chat to get advice on _how_ to commit a crime?
Yes it should be, but it should be illegal for a company providing chat service to respond with anything other than a refusal when doing so.
That detection and refusal should be a private closed loop though, anonymizing any data that will be passed into a training pipeline, or ads targeting. This requirement for closed loop private chats should be mandated by law sooner than later. Otherwise we're getting into very tricky territory where the temptation of alerting on things like pre-crime grows too close.
Philip K. Dick, Minority Report, 1955
While I accept that this sort of thing is well with in the ToS and regular course of business of any major online platform, it hits different coming from an AI company for some reason.
It should catch normal people becoming unstable so that they can receive help. It is just highly unfortunate that the US legal system works in ways where now this woman's name is public.
___
Of course, we also want purely private tech, but that needs a certain level of merit and sanity filter.
Do we, though? What if someone started an AI company that uses end-to-end encryption to make it impossible for anyone but you to access your data? Personally, I would switch to it in a heartbeat assuming it's competitive with the other products. I don't think it's the tech companies' job to surveil the population and prevent crimes. That said, I'm not necessarily against Anthropic or other companies reporting suspicious activity if their existing systems are detecting it. I'm just not sure we want every product to be forced into that data model.
Your follow-up about purely private tech seems to contradict your first statement. We can either have privacy or surveillance, not both.
That is correct! And exactly my point.
We want both, but, on paper, that is impossible. But in reality, we make it sorta mostly happen anyway, through making the easy defaults not private, and the private stuff not easy.
This is not ideal, because [various reasons I do not need to tell you], but it has proven to be the best we can do to mostly achieve both goals.
Kinda like how capitalism isn't great but just the least worst option we've found so far.
___
The actual fundamental underlying problem being that not all people are equal, but we kinda have to pretend they are, because not doing so leads to fascism and other terrible stuff.
But we kinda also do not want to fully pretend that, because doing so leads to yet other terrible stuff.
Hence the quadruple-speak and contradictions to kinda sorta somehow have a somewhat functioning reality.
A lot of people causing issues are people that can't make sense of many things (like many terrorists). They get a fixed idea and they end up doing something bad. You would catch those with some (basic) surveillance.
A lot of normal people (not wanting to cause issues) might benefit from some privacy, if they understand what are the trade-offs (like government overreach). They can then use a slightly more complex tech.
We would still remain with the couple intelligent but sociopaths (think Unabomber style), but I think no solution can fix all cases.
Why wait for a company to build it? Get your own local hardware like I did and have those guarantees because YOU set it up.
Even if open models were competitive, it's still typically going to be more expensive than a cloud provider because of low utilization and higher purchase price.
You use benchmarks, you test, and because YOU'RE the sysadmin you know what weights are running at what time, it's very visible. You can airgap the hardware and be guaranteed it won't change over time. And, frankly, degradation over time doesn't seem to be what's happening with the open models.
There are trade offs. ISP effectively is like driving on a highway, everyone can see where you are going but not what is inside the car. Id like these AI chats to be the same but they are not.
LLMs aren't email or file storage. AI labs aren't just shuttling bytes around, they're interpreting those bytes and taking action based on them. These models _already_ react viscerally in response to users saying disturbing things: the only practical difference is the ability (or obligation) for the model to escalate that concern. I'm not sure the ethics we hold AI companies to should be different than if a human being was typing out the responses.
Privacy is obviously hugely important, but this isn't the government surveiling every message. It's companies having an obligation to flag real, credible threats according to the law, which is a very different problem space.
Back to LLM chats: A system that can declare her "unstable" is also one that can permaban you from all air-travel because you "privately" said unflattering things about Dear Leader.
I urge those people to share their full information from banks, companies (salaries, agreements, contracts), full health information and share publically all the texts they ever wrote (incl. as teenagers) and all the photos they have taken. And give away all the passwords to all of the services so people could check that they are truly clean. Just to be sure, just for everyone's security, right?..
In a old happy little idealised village, it became known quickly, who started to behave oddly and timely intervention could happen. In the modern anonymous mass cities?
No one (wants to) notice the madmen scheming in his isolated flat, surrounded by strangers. Until he explodes.
Unfortunately I also don't trust our government agencies with the surveillance - because they ain't transparent either and the self surveillance seems broken.
" "I'm going to kill that guy", which for non-autists means "that guy was really annoying" you should be reported."
And unfortunately there are lots of real threats being made under the disguise of humor. And much harder to separate im text. So maybe don't talk of murdering people in general, AI surveillance or not?
But in this reality, sure, rather surveil everyone with a baseline level and surveil dangerous ones gradually more. This is roughly how it works today .. just not very good. And with too many exceptions for the powerful.
----
jfc, why is this website full on psychopaths
"The actual fundamental underlying problem being that not all people are equal, but we kinda have to pretend they are, because not doing so leads to fascism and other terrible stuff.
But we kinda also do not want to fully pretend that, because doing so leads to yet other terrible stuff."
other people are not pretending everyone is equal. your power levels are showing, it isn't subtle.
Normal people live prejudice. It's (erm, claude-speak) load-bearing for them, given just how complex reality is and given just how well it reduces that complexity.
So... minority report?
Maybe you do; I want my tech to always include secure, encrypted communications. Don't include me in your destruction of privacy with your silly bandwagon!
Then again, I wish this worked in a way that would give users more privacy and agency, instead of less.
I shot the sheriff
Lie down you bastard!
But I did not shoot no deputy nooo....I would think a good lawyer could get this charge dropped. I would like to see what judge approved the warrant and how they felt the elements were met.
Two teens riding in a Waymo were arrested because the AI detected them talking about having a gun.
Basically, under this interpretation, any personal note you store in the servers of a company could qualify, even if you didn't ever imagine someone would read and as such you couldn't have thought about it as a threat
> The arrest report states that on Sept. 26 at approximately 5:10 a.m., Heller reportedly wrote, "I'm going to shoot up the sheriff's right the [expletive] now." The following day, at approximately 1:07 a.m., she was accused of posting, "This is 100% last chance I'm done. I got a new [expletive] gun today. [Expletive] you."
[1] https://www.gulfcoastnewsnow.com/article/florida-woman-arres...
1791144575 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49957340 | 0 comments
1791147034 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49957692 | 0 comments
1791149399 | Florida woman used Claude as a diary, then Anthropic reported an entry to police | https://www.techspot.com/news/114091-florida-woman-used-clau... | https://news.ycombinator.com/item?id=49958089 | 3 comments
1791213096 | Florida woman arrested for allegedly making threats in an AI chat | https://www.theverge.com/ai-artificial-intelligence/1004747/... | https://news.ycombinator.com/item?id=49965895 | 2 comments
How does a LLM prompt satisfy this? I guess it'll be an easy win for her.
To me this is just straight up thought crime, they just don't have a way to directly read your thoughts yet. But they will spy on you and try and catch you out for it.
IANAL but it will be interesting to see how the legal system decides if this counts as "another person may view it" or not. What has happened in similar ish cases where someone writes a threat like that that they thing is private to them but actually ends up in the hands of Someone at some tech company that reports it?
I think the defendant could successfully defend themselves by claiming they did not know (or intend!) the message could be viewed by another person, as they were plainly using it as a private diary.
People used to say that Facebook knew more about you than your closest friends/relatives. And compared to that, this is just on a completely different level. Absolutely insane.
Anyway, I'm gonna continue avoiding sending anything I can through servers controlled by a company - especially American tech companies - without some sort of end-to-end encryption with an intended recipient. I would highly recommend that every living person do the same. Obviously it's quite difficult to avoid completely, but given the capricious nature of both governments and the general public in what they're willing to come after people for, I'd rather take as few chances as possible
The way I described AI when I first encountered it was "Google on steroids", and honestly it hasn't proven me much wrong. It takes all the results we used to find on Google, and generates code from it, so what? It's still just Google on steroids to me.
Don't google "how to hide a body", just as well as you shouldn't ask your AI how to hide a body. Not much has changed overall.
https://www.politico.com/news/2023/08/30/desantis-warns-hurr...
The downside is that you don't get cached prompt discounts, so you pay a heavy price for ZDR that way.
I’ve had them email me before because I was testing it as a filter for abusive messages and they detected some no-no and wrongthink in those test messages.
Obviously, as others have pointed out if they don't act and she does the crime it raises the damned if you do damned if you don't. But I've also had the AI's go haywire saying I'm doing all sorts of nefarious things when literally doing math proofs.
So no one size fits all. But going the extreme first is probably not ideal.
Seems to fail this test at face value.
I mean, somebody you live with may find and read your diary. Is that the same thing?
Intent matters here. Did you intend somebody else to view it? Does a reasonable person have expectation of privacy with a chatbot?
Every single lie of yours is exposed in the past few months.
You get privacy if you're a big corporation that needs to make sure OpenAI/Google/Anthropic can't read your trade secrets etc.
But those contractual privacy protections have been in place for a long time. It doesn't have anything to do with AI, it's been the same with Office365, Google Docs, etc.
LocalLLM enthusiasts exist for a good reason.
For example, if you said to your counselor or psychologist that John Smith is going to rob the bank next Thursday, they will report that to prevent the imminent risk of serious harm to others.
And remember that anything but local AI you personally control is not anonymous. Not even AI you pool with your friends.
Can the public also immediately get alerted when a cop or politician does some bad shit, though?
And here we all are, happily typing our stuff into these spy chatbots. "AI" happily made our fears go away. Hopefully we're not planning anything criminal like this woman, but still.
Snitching on the people - good mass surveillance company.
On the other hand, people need to learn to not trust these companies. It reminds me of others being surprised when a self-driving car reported a gun in the car. I mean, do people not think? Besides, of course, it's already messed up to want to have a gun. And it is constantly one country that has such issues, more so than many other countries.
Thought crimes are real when you're sharing your thoughts with Claude
Summary: don't type in Claude anything you wouldn't like a human to read.
I don't think someone is an idiot for thinking that the information they type into their private Claude account is private. I also don't think people are idiots for thinking their phone is listening to them and giving them targeted advertising based on that. Both are reasonable deductions from their lived experiences. Both are wrong.
https://www.nbcmiami.com/news/local/everyone-deserves-to-die...
https://www.wdsu.com/article/maryland-high-school-student-ch...
https://www.pinellassheriff.gov/21-023-deputies-arrest-pinel...
If it were written on paper, and only in a room with no phones or cameras so fable couldn't hack it, then I think you wouldn't be sharing it.
I think it’s valid to ask if tapping something into Claude is legitimately sharing a threat.
I don’t think it is. I also think the sheriff could have found more-substantial evidence if she was actually planning domestic terrorism.
That said, if the shooting happened and we were looking at this from before? It’s a tough balance without an easy answer.
I think it’s fair to pre-identify folks who fantasise about shooting anyone. It’s a small fraction of the population that looks into logistics versus making offhand comments.
Claude: Law enforcement has been notified, you are now under arrest.
Imagine if, instead of a friend, he had asked an AI chatbot. Would he have been reported to the police?
Aren't our private thoughts just that, private?
I get that a cloud-hosted AI chatbot is, to tech-savvy people, immediately "not private", but a lot of people don't understand this. What if Stephen King was talking to his friend about planning the murder, and his phone was listening? What if he was typing it up in MS Word and the program decided to phone home and report him to the police?
Nope! Another incident has taken place that illustrates how committed Anthropic and Anthropic AI is to the law and public safety, unlike all that open model riff-raff
They were waiting with bated breaths for something like this to jump on, and make an example of.