They get friendly and loose-lipped with the bartender over the span of months. Eventually they let slip that they plan on killing their spouse for a life insurance payout. At first the bartender thinks they're joking, but it becomes evident that there's an actual plan being acted upon and someone's life is very likely in imminent danger.
Does the bartender have a responsibility to go to the police?
/s obviously
It's not 2005 anymore. If you think that there isn't any way for the people operating an online service to surface your activity on that service, or that there is but those people aren't doing so, there might not be anything left to convince you of it.
What you put into a text box online can be used against you. Period. You have to act accordingly.
If you are a business and I am using your services, it is pretty damn unethical and wrong to vacuum up my data and hand it off. Yes I know they all do it, I’m not naive. But a lot of comparisons people are making are not analogous to straight up sending a message or saying something to someone directly. Companies thrive on opacity and convoluted EULA’s to spy on us without clearly saying they are. Please do not talk down to me just because I’m talking about how things should be, about what is right and wrong, rather than blindly going “well AkShOoAlY you signed the thing and you should know that everyone is always trying to screw you so just live accordingly.“ I’m not OK with that, that is not how I want to live my life even if I am forced to, and I am going to make it known that I take issue with it.
Corporate surveillance is a blight, it is literally harming our society. Every time you tell people essentially “deal with it” you are reinforcing the current situation. Expect better from companies and society as a whole. Demand better.
You are clearly a smart person and you want to have a discussion, so is this the argument you want to make? Effectively defending companies by telling people to just suffer their abuses or keep their heads down?
But if you feel so moved to use the product or service, or must for some reason, well... don't mention that you're thinking about committing a crime.
Also, I'd say that this isn't quite as clear as vacuuming up people's data and handing it off. There's not much money in calling the police on your users. What there is, is reduced liability.
Preface: IANAL.
Let's say the user in the story told Claude (or whatever product of Anthropic's she was using) that she was thinking of attacking law enforcement, and actually carried out the attack. People are injured or killed. Facilities are damaged. She's either arrested or killed. Either way there's an investigation into what happened. Law enforcement looks at her digital footprint and sees that she had mentioned to Anthropic through Claude that she was thinking about committing the crime.
Since there's now loss involved - the health of the survivors, the lives of the dead, and the usability of the facility - there's now a search on how to recoup that loss monetarily. The lawyers file a suit alleging that Anthropic neglected their duty of care to their customers and the community, because supposedly Anthropic's worth hundreds of billions (if not trillions) of dollars and those bastards need to pay.
The typical legal shenaniganning by Anthropic's lawyers doesn't get the suit tossed and it goes to trial. Survivors talk about how their lives have been changed forever, and opposing counsel says it's because those damned tech bros with the data centers and the job-killing AI didn't report the threat to law enforcement. The jury, being made up of people from the general public (who are polling as antagonistic towards AI [0]), agrees. They award damages.
Other lawyers, having case law and a template for victory, smell blood in the water. They put out the "if you or a loved one" ads like they did for asbestos and tobacco companies, but for AI. Individual suits and class-actions are filed.
Now the company is facing potentially billions of dollars in judgments and investors are looking for the exit because, well, there's not any money in paying off legal judgments.
And that's before we take into account any regulations that get passed.
If you're Anthropic's corporate counsel, and you're faced with this possible scenario, you tell their C-suite that you will quite literally lunge over the table and throttle them if they don't implement a policy to inform law enforcement of possible threats found in chats.
[0] https://news.gallup.com/poll/712751/americans-cool-toward.as...
Good luck going through life without touching Google, Facebook, etc. You don’t even need to deliberately use them, they’re just everywhere. I have extensions and more trying to block their trackers every damn day. The amount of work I put into protecting my identity and data is insane and I know it’s still not enough. But here’s the relevant part and what I’m surprised I need to say: We can’t possibly expect the broader population to exert the same effort. It’s not reasonable to expect individuals to take on decades old near-trillion-dollar companies that are literally dictating legislation. Caveat emptor does in fact have limits. It’s not a fair fight at all.