Palisades Fire suspect's ChatGPT history to be used as evidence
rollingstone.com
rollingstone.com
In reality, Uber records and conflicting statements incriminated him. He seems to be the one who provided the ChatGPT record to try to prove that the fire was unintentional.[1]
> He was visibly anxious during that interview, according to the complaint. His efforts to call 911 and his question to ChatGPT about a cigarette lighting a fire indicated that he wanted to create a more innocent explanation for the fire's start and to show he tried to assist with suppression, the complaint said.
[1] https://apnews.com/article/california-wildfires-palisades-lo...
So they probably have developed the tool, and once developed been secretly compelled to use it.
My understanding is that Apple’s executives were surprised at the forcefulness of the opposition to their stand together with the meekness of public support.
(Having worked on private legislation, I get it. You work on privacy and like two people call their electeds because most people don’t care about privacy, while those who do are predominantly civically nihilists or lazy.)
https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...
If Apple had simply had the text records, they would have had to comply with the government order to provide them.
Not Mullvad. Swedish police showed up looking for some dat, Mullvad didn't even collect what they wanted, police left empty handed.
The specifics vary by country, but basically all legal systems require you to comply with what they say and impose penalties if you don't. I don't know if there are any countries where it's legal to ignore the courts, but I would imagine that their court systems don't work too well.
See Wilson v. United States
Tech is full of people that make extremely good money, from other people's personal information, and they plug their ears and sing "La-la-laaaa-I-can't-hear-yooouuu-la-la-la", when confronted with information that says what they are doing has problems. Not just techhies. That's fairly basic human nature.
This is pretty much the embodiment of Upton Sinclair's quote: "It is difficult to get a man to understand something, when his salary depends upon his not understanding it."
For my part, I don't collect any data that I don't need; even if it makes it more difficult to do stuff like administer a server.
> Investigators, he noted, allege that some months prior to the burning of the Pacific Palisades, Rinderknecht had prompted ChatGPT to generate “a dystopian painting showing, in part, a burning forest and a crowd fleeing from it.” A screen at the press conference showed several iterations on such a concept...
Video here, including the ChatGPT "painting" images circa 1m45s: https://xcancel.com/acyn/status/1975956240489652227
(Although, to be clear, it's not like the logs are the only evidence against him; it doesn't even look like parallel construction. So if one assumes "as evidence" usually implies "as sole evidence," I can see how the headline could be seen as sensationalizing/misleading.)
There is text input and text output it's really not that complicated
If used in court the jury would be given access to the full conversation just like if it was an email thread
Presumably what's of evidentiary value is the tokens you type, though.
https://openai.com/index/response-to-nyt-data-demands/ (yes, that's written 100% from OpenAI's perspective)
In particular:
> The New York Times is demanding that we retain even deleted ChatGPT chats and API content that would typically be automatically removed from our systems within 30 days.
> ...
> This data is not automatically shared with The New York Times or anyone else. It’s locked under a separate legal hold, meaning it’s securely stored and can only be accessed under strict legal protocols.
> ...
> Right now, the court order forces us to retain consumer ChatGPT and API content going forward. That said, we are actively challenging the order, and if we are successful, we’ll resume our standard data retention practices.
I'm sure that there are many people who thoughtlessly type very personal things into chatgpt including things that might not look so good for them if they came out at trial.
As I understand it, some people treat chatgpt like a close personal friend and therapist. Confiding their deepest secrets and things like that.
Same difference as: "Allowing minors into casinos... is it any different from letting them play cards with their friends at home with their pocket money?"
I take issue with the "is designed to" phrase. That implies an intentionality upon OpenAI (and others) to create something that acts as a therapist or confidant. It is designed to respond to you in a way that you ask it to. The agency for confiding deep secrets to a cloud service is entirely upon the human typing in the text.
If one doesn't try to make it your friend, it doesn't try to act like it.
[1] - https://web.archive.org/web/20251008204636/https://www.rolli...
(and yah, yada yada about journalism no longer, or maybe never, being about truth, I get it, but still IMO the field should be held to the higher journalistic standard)
It's no different than the contents of your home. Obviously we don't want police busting in to random homes to search, but if you're the suspect of a crime and police have a warrant, it's entirely reasonable to enter a home and search. I guess it can't necessarily help clear you up like an alibi would, but if the party is guilty is could provide things like more certainty, motivation, timeline of events, etc.
I think people conflate the two. They hold that certain things should remain private under all circumstances, where I believe the risk is a large dragnet of surveillance that affects everyone as opposed to targeted tools to determine guilt or innocence.
Am I wrong?
We’ve long ago entered a reality where almost everyone has a device on them that can track their exact location all the time and keeps a log of all their connections, interests and experiences. If a crime occurs at a location police can now theoretically see everyone who was in the vicinity, or who researched methods of committing a crime, etc. It’s hard to balance personal freedoms with justice, especially when those who execute on that balance have a monopoly on violence and can at times operate without public review. I think it’s the power differential that makes the debate and advocacy for clearer privacy protection more practical.
Plenty of big services will just give cops info if they ask for it. It's legal. Any company or individual can just offer up evidence against you and that's fine, but big companies will have policies that do not require warrants.
Despite this atrocious anti-privacy stance, cops STILL clear around half of violent crimes, and that's only in states with rather good police forces, usually involving higher requirements than "A pulse" and long training in a police Academy. Other states get as low as 10% of crimes actually solved.
When you've built a panopticon and cops STILL can't solve cases, it's time to stop giving up rights and fix the cops.
I think this is where policy is failing. No clear protections on privacy and collusion between corporations and the state is allowed. It’s outdated and impractical to have the limits on search and seizure at physical boundaries but not electronic ones.
In other words: if it's Joe Schmoe's Haberdashery forwarding CCTV footage to police to elucidate a crime right in front of their door, sure, it's fine and dandy, they do have an interest in not having crime in front of their door. But when Revolving Door MegaCorp builds a dragnet of surveillance AND is also selling cloud contracts to the government by the billion, it becomes a lot more murky if they just start snitching on everything they see.
As a naturally curious person, who reads a lot and looks up a lot of things, I've learned to be cautious when talking to regular people.
While considering buying a house I did extensive research about fires. To do my job, I often read about computer security, data exfiltration, hackers and ransomware.
If I watch a WWI documentary, I'll end up reading about mustard gas and trench foot and how to aim artillery afterwards. If I read a sci-fi novel about a lab leak virus, I'll end up researching how real virus safety works and about bioterrorism. If I listen to a podcast about psychedelic-assisted therapy, I'll end up researching how drugs work and how they were discovered.
If I'm ever accused of a crime, of almost any variety or circumstance, I'm sure that prosecutors would be able to find suspicious searches related to it in my history. And then leaked out to the press or mentioned to the jury as just a vague "suspect had searches related to..."
The average juror, or the average person who's just scrolling past a headline, could pretty trivially be convinced that my search history is nefarious for almost any accusation.
DAs for bigger departments are likely well equipped, well trained, and well practiced at tugging on the heartstrings of average juries, which are not average people, because jury selection is often a bad system.
The modern abuse of the third-party doctrine is a different topic. Modern usage of the third-party doctrine claims (for instance) that emails sent and received via Gmail are actually Google's property and thus they can serve Google a warrant in order to access anyone's emails. The old-timey equivalent would be that the police could subpoena the post office to get the contents of my (past) letters -- this is something that would've been considered inconceivably illegal a few decades ago, but because of technical details of the design of the internet, we have ended up in this situation. Of course, the fact there are these choke points you can subpoena is very useful to the mass surveillance crowd (which is why these topics get linked -- people forget that many of these mass surveillance programs do have rubber-stamped court orders to claim that there is some legal basis for wiretapping hundreds of millions of people without probable cause).
In addition (in the US) the 5th amendment allows you the right to not be witness against yourself, and this has been found to apply to certain kinds of requests for documents. However, because of the third-party doctrine you cannot exercise those rights because you are not being asked to produce those documents.
In the past, if you put evidence in a safe and refused to open it, the police could crack it, drill it, cut it open, etc. if all else failed.
Modern technology allows wide access to the equivalent of a perfectly impregnable safe. If the police get a warrant for your files, but your files fundamentally cannot be read without your cooperation, what then?
It comes down to three options: accept this possibility and do without the evidence; make it legally required to unlock the files, with a punishment at least as severe as you're facing for the actual crime; or outlaw impregnable safes.
There doesn't seem to be any consensus yet about which approach is correct. We see all three in action in various places.
1. How wide is the search net dragged?
2. Who can ask for access?
The first shows up in court cases about things like "which phones were near the crime" or "who in the area was talking about forest fires to ChatGPT?" If you sweep the net far enough, everyone can be put under suspicion for something.
A fun example of the second from a few years ago in the New York area was toll records being accessed to prove affairs. While most of us are OK with detectives investigating murders getting access to private information, having to turn it over to our exes is more questionable. (And the more personal the information, the less we are OK with it.)
Do you think OpenAI wont produce responsive records when it receives a lawful subpoena?
And very rightly so, regardless if Uber records incriminated this person.
If a person is suspected of committing a crime, and police obtain a specific, pointed, warrant for information pertaining to an individual, tech companies have a moral obligation to comply, in the best interests of humanity.
If law enforcement or spy agency asked for a dragnet warrant like "find me all of the people that might be guilty of XYZ" or "find me something this individual might be guilty of"; tech companies have a moral obligation to resist, in the best interest of humanity.
The first is an example of the justice system working correctly in a free society; the second is an example of totalitarian government seeking to frame individuals.
Sounds like they got the info from his phone, not taken from any servers, so this is likely not an example of a tech company "complying".
I'm not taking sides, but it sounds like if ChatGPT cooperating with LE is a Good Thing (TM), then Apple making a public spectacle of how they are not going to cooperate is .. bad?
I'm fully aware that Apple might not even be able to provide them the information, which is a separate conversation.
And note: the above scenario is not likely anywhere in the near future, because homomorphic encryption has something like a million times overhead, and requires you to hit the entire database on every request, when state-of-the-art LLM systems are already pushing the limits of computation.
My understanding is the suspect was detained and law enforcement was not asking for a dragnet (at least thats what they stated publicly), and they were asking for a tool for a specific phone. Apple stated the FBI was asking them to backdoor in all iPhones, then the FBI countered and said thats not what they were asking for. Apple then marched triumphantly into the moral sunset over the innocent victims'; meanwhile the FBI then send funds to a dubious group with questionable ethics and ties to authoritarian regimes.
In my opinion, Apple should have expediently helped here, if for no other reason than to prevent the funding of groups that support dragnets, but also out of moral obligation to the victims.
Instead I am only aware from what has been published that there is the so-called "Secure Enclave" chip in the iPhone hardware manifest that will only give up its secrets to a biometric match, or a user password. That would seem to leave Apple's hands tied?
Law enforcement at the time was even admitting (which we'd later find out to be correct) that there likely was nothing of value on the phone. It seems fairly obvious that the FBI was trying to use a high profile case to force a paradigm shift. Perhaps we can argue it'd be a good and just one, but arguing that they weren't seems not right.
There is no world in which a post-PRISM compliant Apple cannot be coerced by the feds for an investigation. It's just a matter of how much pressure the FBI wanted to apply; Apple's colossal marketing win is the sort of thing that you would invent if you wanted to manufacture consumer trust, not "prove" anything to cryptographers. Playing devil's advocate, "authoritarian regimes" are exactly the sort of place you would send the iPhone to if you already had the information and wanted to pretend like it was hard to access.
If we assume a worst-case-scenario where Apple was already under coercion by the FBI, everything they did covers up any potential wrongdoing. It was all talk, no walk. Neither side had to show any accountability, and everyone can go on happily using their devices for private purposes.
See: https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...
>Most of these seek to compel Apple "to use its existing capabilities to extract data like contacts, photos and calls from locked iPhones running on operating systems iOS 7 and older" in order to assist in criminal investigations and prosecutions. A few requests, however, involve phones with more extensive security protections, which Apple has no current ability to break. These orders would compel Apple to write new software that would let the government bypass these devices' security and unlock the phones.[3]
That's much more different than OpenAI dumping some rows from their database. If chatgpt was end-to-end encrypted and they wanted OpenAI to backdoor their app I would be equally opposed.
I agree, the line is at messing with End to End Encryption. If your E2EE has a backdoor ITS NOT END TO END ENCRYPTION. Thanks.
Apple also tried to freak the public out saying the FBI wanted a backdoor added, which was inaccurate. You can't retroactively add a backdoor, that's the whole point of it. FBI wanted Apple to unlock a specific phone, which Apple said they were capable of doing already.
Edit: ai has already passed the bar exam.
My personal believes (we should not rely on models for such things at this stage, let's not anthropomorphize, etc.) to one side, let me ask, do you think if I used my friend Steve, who is not a lawyer but sounds very convincingly like one, to advice me on a legal dispute, that should be covered by attorney client privilege?
Cause, even given the scenario that LLMs suddenly become perfectly reliable enough to verifiably carry out legal/medical/etc. services to a point where they can actually be accepted into day-to-day practice by actual professionals and the companies are willing to take on the financial risks of any malpractice for using their models in such areas (as part of enterprise offerings for an extra fee of course), that still wouldn't and shouldn't mean that your run-of-the-mill private ChatGPT instance has the same privileges or protections that we afford to e.g. patient data when handled digitally as part of medical practice. At best (again, I dislike anthropomorphizing models, but it is easier to talk about such a scenario this way), a hypothetical ChatGPT that provides 100% accurate legal information would be akin to a private person who just happens to know a lot about the law, but never got accredited and does not have the same responsibilities.
Again though, we are far from that hypothetical anyways, "people" using LLMs that way does not change this fact. I know, unfortunately, there are people who are convinced that current day LLMs have already attain Godhood and are merely biding their time and that doesn't become real either, just because they act according to their assumptions.
I really struggle to understand, nor do I see any cogent arguments across this comment section why current day LLMs in such a scenario should be treated differently to e.g. a PKM software or cloud hosted diary and afforded the same legal protections (or lack thereof depending on viewpoint, personal stance and your local data privacy laws).
Protectionism hurts all of society to benefit a few.
Milton Friedman explains it to doctors here: https://m.youtube.com/watch?v=ss5PxPlnmFk
Don't know where to start, but I want to assure you, no matter where on this planet you live, Medical Doctors are generally not at fault for high costs of care. Depending on which health care system we are talking about, the particulars may be different, but no, MDs are not interested in worsening patient care for their own benefit. Kinda difficult considering the amount of uncompensated labor and stress compared to other higher paying occupations. Ask a trainee/resident/equivalent for your local health care system if you want some details.
And people are "forced" to go to an MD for medical treatment in the same way they are "forced" to go to any other domain specific expert, it is where the experience and liability lie because they have undertaken the time, training and exams to ideally assure a specific level of care.
Incidentally, has absolutely zero to do with LLMs and the fact that this is cloud hosted software, not an entity, being or anything of the sort, so shouldn't receive any special considerations beyond what we afford to cloud hosted content. Couldn't find anything on patient data processing in that MF collection linked and as that was his area of work, was purely US centric. Medical care is however the purview of medical professionals outside the US as well, including in countries with far higher patient outcomes. If there is an applicable argument, just quote that directly over linking a collection of clips.
To bring this back to the topic at hand, LLMs can and are being used in Medical Practice already. And neither did Doctors prevent that, nor did that require a law change, because, as stated before, it is merely data being input and processed. There are EU MDRd apps for skin cancer, there are on-prem LLM solutions that adhere to existing patient privacy regulations, etc.
Basically, Doctors do not stand in the way of LLM usage (neither could they, nor do they have the time) and even if they wanted to, LLM input and output is just data and gets treated accordingly.
All you assured me of is that you didn't watch the video.
Don't expect that from products with advertising business models
Real research is done on prior art. We research things because prior studies tell us that they are feasible, otherwise you are wasting valuable time on a snipe chase. There is no reproducible or substantial evidence that AGI or singularities exist. It is another Big Tech marketing lie, no different from the reneged "don't be evil" or "privacy is a human right" mottos.
If your business model is theoretical, we call that "research" instead.
There is no reproducible or substantial evidence that AGI or singularities exist
Regardless: tell them, not me! I'm just reporting what I'd say is an objective fact: they are planning based on scientific predictions of an intelligence explosion -- at least a soft/cybernetic one if not a scarily-fast/purely-digital one.
It is another Big Tech marketing lie
They very well may be wrong of course, but I think you're doing yourself a disservice to assume they're lying about it.
You are wasting my time with facetious arguments. There is no point having a rational discussion about the future potential of AI if we cannot take things like reality for granted.
If you want to argue in defense of AI, do it. Pointing to authority is one third of rhetoric, the other two thirds are emotional investment and logical coherence. If you don't have real proof that AGI exists, you're trying to make a point with emotions that people don't empathize with and authority that isn't authoritative. Cite sources, dammit.
the boosters are getting desperate
Now the real question is, do you consider it a conversation or a letter. Any opened¹ letters you have lying around at home can be grabbed with a court-ordered search warrant. But a conversation—you might need the warrant beforehand? It's tricky.
(Again, exact legal situation depends on the country.)
¹ Secrecy of correspondence frequently only applies to letters in sealed envelopes. But then you can get another warrant for the correspondence…
For the LLM input, that maps 1:1 to documents a person has written and uploaded to cloud storage. And I don't see how generated output could weigh into that at all.
The question you are asking is: should I approach my daily search tasks with the same degree of thoughtfulness and caution that I do with my document storage choices, and do I have the same options? And the answers I would give are:
* As a consumer I don't want to have to think about this. I want to be able to answer some private questions or have conversations with a trusted confidant without those conversations being logged to my identity.
* As an OpenAI executive, I would also probably not want my users to have to think about this risk, since a lot of the future value in AI assistants is the knowledge that you can trust them like members of your family. If OpenAI can't provide that, something else will.
* As a member of a society, I really do not love the idea that we're using legal standards developed for 1990s email to protect citizens from privacy violations involving technologies that can think and even testify against you.
Then treat them with the same degree of thoughtfulness and caution you have treated web searches on Google, Bing, DuckDuckGo or Kagi for the last decade.
Again, there is no confidant or entity here, no more so than the search algorithms we have been using for decades are at least.
> I really do not love the idea that we're using legal standards developed for 1990s email to protect citizens [...]
Fair, but again, that is in no way connected to LLMs. I still see no reason presented why LLM input should be treated any differently to cloud hosted files or web search requests.
You want better privacy? Me too, but that is not in any way connected to or changed by LLMs being common place. Same logic I find any attempt to restrict a specific social media company for privacy and algorithmic concerns laughable, if the laws remain so that any local competitors are allowed to do the same invasions.
The closest I can find is "keyword warrants" where police ask for users who searched on a given term, but that's not quite the same thing as an exhaustive search history.
Certainly my personal intuition is that historically there has been a lot of default privacy for non-logged in "incognito" web search, which used to be most search -- and is also I think why we came to trust search so much. I expect that will change going forward, and most LLMs require user logins right from the jump.
As far as the "I can see no reason" why LLMs should be treated differently than email, well, there are plenty of good reasons why we should. If you're saying "we can't change the law," you clearly aren't paying attention to how the law has been changing around tech priorities like cryptocurrency recently. AI is an even bigger priority, so a lot of opportunity for big legal changes. Now's the time to make proposals.
A massive amount, part of why I am both surprised and starting to feel like this discussion stems from some being unaware of the tracking they tolerated for decades. These have been discussed to no end, covered by the usual suspects like the EFF and constantly get (re)reported across the media in "Incognito mode is not incognito" pieces.
Heck, some I know from memory [0], the rest one could find with a simple ten sec search [1].
> [...] my personal intuition is that historically there has been a lot of default privacy for non-logged in "incognito" web search [...]
There has not. No need for intuition or to believe me, just read the privacy information Google provides [2] whenever you access their sites (whether in an incognito instance or otherwise) as part of the cookie banner (and in the decade beforehand if one looked for it).
> As far as the "I can see no reason" why LLMs should be treated differently than email, well, there are plenty of good reasons why we should.
Not email. Never said email. If you are going to use quotation marks, please quote accurately ("I still see no reason presented why LLM input should be treated any differently to cloud hosted files or web search requests." is what I wrote and means something very different), I do the same to you.
Neither you, nor anyone else has provided a reason why LLM input is inherently different to other files hosted online. Happy to read those "plenty of good reasons", but they have yet to be shared.
> If you're saying "we can't change the law," [...]
I did not. I asked why existing laws should be applied differently in case of LLM input and/or changes are somehow needed for LLMs specifically or suddenly.
This really seems like LLMs, because they can be anthropomorphized, "feel" different to some and that somehow warrants different treatment, when that is an illusion.
Considering your believe that "historically there has been a lot of default privacy for non-logged in "incognito" web search", it honestly sounds like you believe there is less room for stricter regulation than my long immersed in this topic self, if I am being fully honest.
If I could implement any change, I would start with more consistent and transparent information of users at all times, which might dispel some misconceptions and help users make more informed decisions, even if they don't read the privacy policy.
Always liked a traffic light system as a concept. Then again, that is what Chrome already tells you when opening incognito mode and somehow there still seem to be assumptions that are not accurate about what that actually does and doesn't do.
TL;DR:
Yes, Search engine providers are able to identify users in incognito mode. Such tracking has always been public information, not least because they have to include it in their privacy policy.
Yes, such tracking has been used in court cases, in the US and elsewhere, to identify users and link them to their search requests done whilst using such modes.
No, LLM input is no different to search requests or files hosted online. Or at least, no one has said why LLM input is different, happy to hear arguments to the contrary though.
[0] https://www.classaction.org/media/brown-et-al-v-google-llc-e... (Google was forced to remediate billions (yes, with a b) of “Incognito” browsing records which according to plaintiffs precisely identified users at the time including being able to link them to their existing, not logged in, Google accounts. Note that this is one of two (US specific) cases I knew of the top of my head, the other was the Walshe murder, though there is no (public) information on whether incognito was used in that case: https://www.youtube.com/watch?v=cnA6XwVQUHY)
[1] https://law.justia.com/cases/colorado/supreme-court/2023/23s... and https://www.documentcloud.org/documents/23794040-j-s10032-22...
[2] https://policies.google.com/privacy?hl=en-US ("When you’re not signed in [...] we store the information we collect with unique identifiers tied to the browser, application, or device you’re using.", "This information is collected regardless of which browser or browser mode you use [...] third party sites and apps that integrate our services may still share information with Google.", I think you get the point. There never was any "default privacy for non-logged in "incognito" web search" and I can assure you, that data has always been more than sufficient to fingerprint a unique user)
But Google’s internal logging is not the question I’m asking. I’m saying: can you find a single criminal case in the literature where police caused Google to disgorge a complete browsing history on someone who took even modest steps not to record it (ie browsed logged out.) Other than keyword search warrants, there doesn’t seem to be much. This really surprised me, since as an expert I “know” that Google has enough internal data to reconstruct this information. Yet from the outside — the experience that matters to people - they’ve managed to operate a product where real-world privacy expectations have been pretty high if you take even modest steps. I think this is where we get many of our privacy expectations from: the actual real-world lived expectations of privacy are much closer to what we want than what’s theoretically possible, or what will be possible in a future LLM-enabled surveilled world.
Can you first point to me making a claim that would require such a case? Or can you, alternatively, point to why there is a need for change rather than just continue to apply the same level of legal protections to LLM service providers?
The fact that this started with a report about a users ChatGPT account and you felt the need to move us towards people using commercially hosted LLMs without an account (cause getting five queries in before OpenAI forces you to sign in is a realistic use case) I let slide up to this point, because whether we are talking about incognito mode access or a user with an account, doesn't change that no one here says why using Chat.com is different to Google.com. I just wanted to call it into memory, cause it is not very expert like, same with the (mis)quoting.
To make it simple, this is my question, the only thing I'd like to have answered:
When self hosted websites first became a thing, governments across the globe did not write new editorial legislation specific for these. They did just apply what was already established for print media/speech.
In this context, why should LLM input be handled differently to data hosted online?
Use file sharing services with no login requirement and the legal requirements there if this completely irrelevant red herring is absolutely necessary for you. Doesn't change anything about the question.
But you do, just like you have confidentiality in what you write in your diary.
There is more evidence they will do this rather than that they won't. ChatGPT is a giant dragnet and 15 years ago I would've argued it's probably entirely operated and funded by the NSA. The police already can obtain a "geofenced warrant" today. We're not more than one senator up for re-election from having a new law forced down our throat "for the children" that enables them to mine OpenAI data. That is, if they don't already have a Room 641A located in their HQ.
People pour their live out into these fuzzy word predictors. OpenAI is holding a treasure trove of personal data, personality data, and other data that could be used for all kinds of intelligence work.
This is objectively bad regardless of how bad the criminal is. The last near 40 years of history, and especially the post 9/11 world, shows that if we don't stand up for these people the government will tread all over our most fundamental rights in the name of children/security/etc.
Basic rights aren't determined by how "good people" use them. They are entirely determined by how we treat "bad people" under them.
Thinking again, the AI will certainly be "free".
I also don't think the interests of the state are "in the best interests of humanity".
Sometimes the price of having nice things and them remaining nice means that people you don't like can use them, too.
The tech companies have just about zero ability to resist.
There should likely be legislation enacted that raises chat logs to the level of psychotherapist-patient privilege.
Medical records are up for grabs when it comes to investigations/discovery/subpoenas/warrants/etc, it's one of the privacy exceptions in HIPAA.
They can also be used to try to identify alleged criminals, missing people, fugitives, etc. For example, if they have DNA samples, bite impressions (even though it's BS), law enforcement can even demand matching medical records without warrants.
Does everyone have the same earnings potential risk regardless of their skill? Same with stealing potential.
Edit: on the flip side, white collar crimes leave a paper trail that traditional smash and grab crimes do not, so more white collar criminals should be getting caught and convicted now.
Human nature doesn't follow your shitfuck ideal driven rules friend. I guarantee some day you'll find that out the hard way.
If you cause a problem, report it, then the authorities responsible for dealing with those problems take care of it and go home, what does it mean?
Are the authorities then partially responsible for not ensuring the fire was put out properly before leaving the area?
Is he even guilty at all given that he filled his duty and reported the problem after unintentionally causing it?
However many US states have a "felony murder rule" which as I understand it says if you did something that resulted in death, and it was in the course of a felony then it can be tried as murder. Most of them rule out some felonies (felony assault + death => murder is a stupid way to apply such a rule and so is usually ruled out) and some only rule in a handful like rape and prison escapes, but felony arson + death => murder might play.
The distinction between murder and manslaughter is malice aforethought. For first degree murder, you must have intended the death of a particular person. For second degree murder, you need only have known that you could kill someone, and did it anyways. This specifically includes things done with extreme recklessness.
So to prove second degree murder you need to show 1) you intentionally did something, 2) you knew (or should have known) it could kill someone, and 3) someone died.
These can be proven for arson. You have to prove the intent to start the forest fire. Everyone knows (or should know) that forest fires can kill people. You have to prove that someone died from the fire.
That is why arson qualifies as second degree murder. Just like, say, failing to maintain the brakes on a fleet of trucks. (True story. My nephew was the unlucky driver of such a truck whose brakes failed...)
The textbook example is running someone over while fleeing the scene of a robbery. You didn't have mens rea for murder, the crime you intended to commit was robbery. But you chose to commit a felony, and someone did die because of it. Not only that, it's potentially capital murder, because it was for financial gain (Newsom put a moratorium on felony murder death sentences, so that's not a thing at the moment).
It is easier to prove the felony murder. Was it on the list of felonies? To prove the second degree murder, you have to demonstrate "extreme recklessness". Prosecutors will often pile up multiple charges like this. To give the jury as many options as possible to convict.
I'm not a lawyer. But in this case the fact that he called emergency services could be evidence against extreme recklessness, and therefore second degree murder. But felony murder still fits.
The case for malicious intent is extremely flimsy and based entirely on circumstantial evidence. The strongest piece of evidence they have for arson is that he threatened to burn down his sister's house but here's the thing, it would be extremely unusual for an arsonist to switch from targeted arson based on anger or revenge to thrill seeking arson setting unmotivated fires.
This is all pet theories and silliness for purposes of discussion. I freely admit that I haven't built a case here that's strong enough to withstand even a gentle poking by an opponent.
I don't know as much about arson but I did go through the same serial killer phase as every morose teen and one of the things that stuck with me is the way that some offenders escalate from simple peeping and stalking all the way up to murder. Another thing that stuck with me is how in some cases when there is an intended victim, esp for revenge, an obsessed mind will often hone in on a single characteristic of the intended victim then transfer victimhood to strangers based on that characteristic. The woman who "wronged" you is a skinny blonde who smokes cigarettes so you go out looking for skinny blondes who smoke cigarettes to victimize in her stead because in your unconscious brain that matches the pattern of behavior that would soothe the wounded entitlement of the offender. Given these facts about the nature of obsessive, vengeance-oriented crime and the fact that the serial killer/arsonist crossover is so common that arson is one of the mcdonald triad of behaviors common to serial killers there's a non-zero possibility that we're seeing a revenge fantasy transferred to another victim. There's also the fact that obsessed criminals tend to want to roleplay or practice and a lot of times their first "serious" crime is one of these roleplay/practice sessions getting out of control. This feels like that to me though I can't prove it. It's like he wanted to see what starting a fire would be like, assumed that the local VFD would get it under control and in doing so would also give him an idea of what the response looked like so he could optimize for escape, then either it got out of control or he tried to inject himself into the emergency response (another common thing among obsessed criminals, many like to relive the crime by being part of the investigation, like to tease investigators by being right under their nose or believe that by injecting themselves into the investigation they can steer it away from them).
Again, does any of this hold up in a court of law? Of course not. Does it hold up in a court of a thread on a post on HN? Maybe, we're here to talk and I'm of a mind that we didn't do anything to fix w/e it was that made people serial killers but there aren't really any serial killers anymore so something must have happened to that behavior. Perhaps stranger arson is a way that the same drivers that led to serial murder before the ~~panopticon~~internet are driving new behaviors now. Intuitively I'm highly confident that the stranger spree killings we see now are driven by those same pressures in a lot of perpetrators and the change in MO is about taking advantage of lag time in law enforcement's ability to correlate facts. Before the internet you could drive a few hours' down the road and start using a new name and unless your old name was already in the system there was basically no way for anyone to know. Obsessed criminals could offend, disappear and wait it out. Nowadays we're really good at ID'ing an offender so obsessive murders have to be one and done, but another strategy could be crimes that are small enough that they don't trigger the kind of dragnet response that involves things like checking all the CCTV cameras in a ten mile circle around the crime and things like that.
edit: everyone seems focused on the "serial killer phase" line that was really intended to be a throwaway. I just mean that I read a lot about them and thought it was shocking and cool to have a "favorite". Gross shit, but I assure you no one was ever in any amount of physical or psychic danger beyond declaring me a pizza cutter (all edge and no real point).
I’m sorry, what? As a former morose teen, I can assure you that a “serial killer phase” is not a universal experience
When I was a kid, some teens who were into darker themes (not all but definitely some) had a phase where they were interested in serial killers. It always struck me more as shallow “edgy teen” posturing than anything else. After Columbine this demographic moved on to other interests, as even a performative interest in real-world violence could lead to official harassment.
Comment OP here: it was exactly this; the safest, most boring possible way to be transgressive. I didn't talk about it as much as other kids who were like this, so I didn't have to stop once people started to actually care/respond, but I did go from keeping my Harold Schechter books on my bookshelf to in a special box in my closet. Merely knowing about these things gave me a little secret thrill like I was some sort of badass with extreme psychic warding able to go into some secret space that most people couldn't stand. In reality I was just desensitized cuz abusive mom and I'm really glad I grew out of it before I got to the part some kids get to where learning isn't enough and they start experimenting.
I had unmedicated bipolar 1 as a teenager. If anyone was going to go through a serial killer phase I would have.
Even as an adult I had some pretty bad episodes prior to being diagnosed early thirties. My brain went some pretty bad, dark places but it never went to serial killer.
I can only hope that you're mistaken on what a serial killer actually is. Mass murderer and spree killing, depraved as they are, have motives that are recognizable by the average person. Serial killer is a special kind of insanity.
The end of leaded gasoline may play a significant role. And/or a reduction in other chemical hazards. Violence was already declining pre-panopticon. https://en.wikipedia.org/wiki/Lead%E2%80%93crime_hypothesis
You did, however, allude to one of my favorite facts about violent crime in America: far from being a cause of violent crime, the rise of violent video games has been correlated with the most dramatic drop in crime in all of recorded history. That's right y'all, it's at least arguable that not only did Doom not inspire violence, it may have actually made us safer.
Electronic data brokers started in the 1950s. The early decades were less insidious but the box was opened. Invasive government electronic surveillance started before Google was founded.
1) this whole case hinges on intentionality and the gov't intends to prove that he set the fire intentionally. part of the chatgpt history is images he generated of fires and people running from fires. If he intentionally set a fire in a wildfire-prone area it doesn't matter that he didn't intend it to be a wildfire or anything he did after he set the fire.
2) If you'd like to have emergency services that are either prohibitively expensive or simply nonexistent, one great way to do that is to make first responders responsible for not doing a good enough job in their responses. I'm honestly not sure what we'd do in cases of blatantly neglectful behavior by a first responder during an emergency response, but beyond intentional malpractice we generally extend an assumption of good faith to anyone who bothers to show up and help during an emergency like this. The first time I get sued for not putting a fire out fast enough or completely enough is the last time I put out a fire.
That's an extreme situation that the gunman put someone in. Imagine it being YOU. Now if you could be the LAST person that gunman ever put in that situation, would you allow them to go to jail forever? Because if that's the case, the number of people in that situation ever again goes from millions to a few thousand over the next 1000 years. And many of those people will REACT and die.
So when someone starts a fire, they were like the gunman. They were willing to let a lot of people die. Then realizing they were wrong, calling the cops, and having them put the fire out, that's the same as the situation as going into 7-11 and aiming the gun, but then putting it down and walking out. But they still risked someone else's life! What if they accidentally slipped their finger? Employee DEAD.
So it's really the same thing. All that being said, I do grant that the waters are muddied at this point with the legal system. The person still deserves to be separated from civil society. He is not CIVIL!
And even though the legal system's waters are muddied, his original actions resulted in 12 people dying. The firefighters that were incompetent are not originally responsible for those 12 deaths.
The reason I want maximum punishment is that it works, it does deter. In this legal system of course there's a 50/50 those 12 people will have died without being avenged at all (and their families - all that are affected), and a 90% chance (if he is found responsible) those 12 people will get this guy in jail for 10 years. And because of those chances, people decide, that fuck even if I'm caught, it seems like in the last 10 years there is a VERY low chance of punishment. Punishment is very important in this world and life. I'm not talking about capital punishment.
A lot of people disagree with all of this, I personally think they have suicidal empathy. They have no empathy to the thousands of people that died from other peoples intentional actions - actions those people KNEW they might end up killing. They have too much empathy for the attacker. It's massive victim blaming.
This is how humans work. We work on probability and approximation. We often act based the consequences of our intentions, not the consequences of our actions.
Someone that learns the consequences of their actions, regrets the harm they inflicted, and changes their behavior as a result, is not the same danger to society they were before. In fact society would be better off reintegrating them because they'll tell others not to do the same thing.
I'm not exactly sure where to fit this in, but people change. A society that makes vengeance the only rule, where death is punished with death, regardless of a person's intentions, is an authoritarian nightmare.
Also, an accident is if a party had fireworks and the fire got out of control. Arson is definitely not an accident if someone dies.
Regarding "people change" argument. I'm not advocating for the death penalty. I'm advocating that we separate non-civil and civil society. If that takes the shape of the next Australia? Sure, then if someone changes, they're not in jail.
I'm also not advocating for someone non-violently stealing bread to be separated from society. Those people can change.
Someone that at one moment of their life decides that someone else's is worthless because they want the contents of a cash register? Remove them from civil society.
People disagree with you not of opinion but because you are factually wrong.
"Evidence shows lengthy prison terms do not have a significant deterrent effect on crime" https://ccla.org/criminal-justice/no-longer-prison-sentences...
"Research Shows That Long Prison Sentences Don’t Actually Improve Safety" https://www.vera.org/news/research-shows-that-long-prison-se...
For example this was going around for a long time:
https://www.brennancenter.org/our-work/analysis-opinion/debu...
Then this happened in 2025 when there was a crack-down on crime:
https://counciloncj.org/crime-trends-in-u-s-cities-mid-year-...
Also, 1% of the population is responsible for most of the crime
https://pmc.ncbi.nlm.nih.gov/articles/PMC3969807/
So if you end up jailing people, the crime just goes down.
Also, then if that's true, then cancel all murder prison sentences!
1. Longer sentences could have no effect on crime rates.
2. Persecuting people for crimes lowers crime rates.
Honestly, to me it reads as "law enforcement is a good idea, prolonged incarceration is questionable".
Reported effects of CECOT on crime
Reduction in crime rates
Since Bukele declared a state of emergency in March 2022 and began mass arrests, El Salvador's crime rates have plummeted.
Homicide rate decline: The country's homicide rate fell from 103 per 100,000 people in 2015 to just 1.9 per 100,000 in 2024, one of the most drastic reductions in recent history.
Increased public safety: Many Salvadorans, long subjected to extortion and violence by powerful gangs, report feeling much safer on the streets.
With all due respect, I did not have any argument.
I was reading your conversation, I had difficulty seeing a contradiction, so I asked a question.
You've introduced one more statement instead, "mass arrests with long-term sentences have drastically reduced crime rate in El Salvador". I see your point but this doesn't really help me with my original question. How do we know that it's the sentence terms is a significant factor (out of the combination), and not the mass arrests or something else? We don't have a control group, do we?
It seems firefighters are not conservative enough when it comes to putting out fires, at least in these 2 cases.
As far as I’ve heard from other articles, he lived in the Palisades at the time and worked as an Uber driver there. He moved to Florida after the fire. This is not very well researched.
a lot of people, especially younger ones, seem to use chatgpt as a neutral third party in every important decision. so it probably has more extensive records on their thoughts than social media ever did. in fact, people often curate their Instagram feeds - but chatgpt has their unfiltered thoughts.
[1] https://www.eff.org/deeplinks/2024/08/federal-appeals-court-...
[2] https://techcrunch.com/2023/12/16/google-geofence-warrants-l...
In this case, there's an explicit middle point - chatgpt.com resolves to a CloudFlare server, so CloudFlare is actually one of the ends here. It likely acts as a reverse proxy, meaning that it will forward your requests to a different, OpenAI-owned server. This might be over a new HTTPS connection, or it might be over an unencrypted HTTP connection.
It really is super important to emphasize this point. End-to-end encryption is not simply that your data is encrypted between you and the ultimate endpoint. It's that it can't be decrypted along the way - and decrypting your HTTPS requests is something that CloudFlare needs to do in order to work.
(To be clear, I'm not accusing CloudFlare of anything shady here. I'm just saying that people have forgotten what end-to-end encryption really means.)
Just for general peace of mind, use a privacy-oriented search engine. I use leta.mullvad.net or search.brave.com usually. I haven't used Google in years. And if you just happen to have a curiosity about something fringe that might be misinterpreted in the wrong circumstances, download an LLM and use it locally.
If you want real and total anonymous search, use a public computer.
https://techcrunch.com/2025/07/25/sam-altman-warns-theres-no...
I have a "saved" history in Google Gemini. The reason I put "saved" in scare quotes is that Google feels free to change the parts of that history that were supplied by Gemini. They no longer match my external records of what was said.
Does ChatGPT do the same thing? I'd be queasy about relying on this as evidence.
1. I engaged with Gemini.
2. I found the results wanting, and pasted them into comment threads elsewhere on the internet, observing that they tended to support the common criticism of LLMs as being "meaning-blind".
3. Later, I went back and viewed the "history" of my "saved" session.
4. My prompts were not changed, but the responses from Gemini were different. Because of the comment threads, it was easy for me to verify that I was remembering the original exchange correctly and Google was indulging in some revision of history.
Rewriting history requires computes which is more malicious. Why would someone burn compute to rewrite your stuff given that rewrites are not free? Once again not defending google trying to think through what's going on.
I can't think of any reason it would make sense to do that, though.
This would be a major tech news story. "Google LLM rewriting user history" would be a scandal. And since online evidence is used in court, it could have significant legal implications. You'd be helping people.
This is much too important to merely be a comment on HN.
Are the 12 deaths separate charges? A sentence of 5-20 years seems very light for 12 deaths. This article is clearly focused on the AI aspect of it, so it doesn't cover the charges at all really.
That seems clear cut first degree murder to me, as I understand it (I'm not sure if it requires a specific person to be murdered but a pre-meditated act that kills people seems like it'd qualify to me).
I think those things are to be decided in court. As for the charges and times, it's mentioned only the ranges for arson but there's nothing to stop them bringing charges of manslaughter for example. They'll build evidence and charge as such. It's the process.
Leaving aside the fact that we don't know yet if he actually started the fire: anyone who starts any fire without appropriate control measures (like extinguishers or containing the fire in something made to contain it) can theoretically be charged under the law for negligence - and practically will, if things go south.
And in a time where there's ample fuel for fires on the ground and the weather conditions are favorable to large fires (e.g. hot, low humidity, clear skies and strong winds) any kind of fire (even smoking - cigarette butts thrown out of car windows are a particularly bad fire source in Croatia) can quickly escalate into a full blown forest fire. Even things that one would not even perceive to be dangerous can cause fires... an all too common occurrence is a diesel car with a freshly regenerated DPF that's being parked on a parking lot that used to be overgrown with weed that's now dried out. The heat from the DPF is massive enough (> 500 °C) to lead to ignition of dried-out weeds (~ 300 °C).
So, it's not a stretch to assume that anyone starting an open fire should know it might escalate into a deadly disaster. And even the reckless cases that I mentioned (smokers, car drivers) can be charged as manslaughter here in Europe.
>Raymond Lee Oyler, 54, of Beaumont, was sentenced to death for starting the Esparanza Fire in October 2006. He was convicted of five counts of first-degree murder, 19 counts of arson and 16 counts of possessing incendiary devices. https://kesq.com/news/2025/05/05/ca-supreme-court-upholds-de...
You may find the "Thin skull rule" interesting for criminal liability
Liability would still be on him.
For those reading: this is the difference between proximate cause and actual cause. Yes it's true that but for the fire being started in the first place, the fire would not have rekindled. But once professional firefighters arrive to put out the fire, it's not foreseeable by a normal person that the fire could be rekindled, so that person wouldn't be liable. The harm is too remote. The firefighters may even be grossly negligent because they are professionals, intervened, and the fire rekindled. A person negligently failing to fully extinguish their own fire would lead to liability, though.
Were not the LA fire occurring in three different locations at the same time? It could not have been started by the same smoldering ash.So they can prove that this was one continuous combustion reaction? They can show beyond reasonable doubt that, despite the observations of the fire going out that convinced a team of firefighting professionals that it had stopped, it in fact continued and nothing else ignited a new fire in this location where fires naturally occur?
1. Adherence to their own customer-facing policy. 2. Corporate or government customers would CERTAINLY want their data handling requirements to be respected. 3. In my experience at $MEGA_CORP, we absolutely delete customer data or never maintain logs at all for ML inference products. 4. They’re a corporation with explicit goal of making money. They’re not interested in assisting LE beyond minimum legal requirements.
But still I wonder what the reality is at OpenAI.
Having a good relation with LE and the state is beneficial to companies, it puts them in a good position to sell them services, get preferential treatment, quid pro quo, etc.
Look at how Google, Microsoft, Amazon etc cozied up to LE and the government. They get billion dollar contracts, partnerships with LE and the more they cooperate willingly, the less likely they'll be sued into cooperation or punished with loss of contracts.
No they can't. People write fiction, a lot of it. I'm willing to bet that the number of fiction related "incriminating" questions to chatgpt greatly numbers the number of "I'm actually a criminal" questions.
Also wonder about hypotheticals, make dumb bets, etc.
If you see CSAM posted on the service then you're required to report it to NCMEC, which is intentionally designed as a private entity so that it has 4th amendment protections. But you're not required to proactively go looking for even that.
End of the day, a chimp with a 3 inch brain has to digest the info tsunami of flagged content. That's why even the Israelis didn't see Oct 7th coming.
Once upon a time I worked on a project for banks to flag complaints about Fraud in customer calls. Guess what happened? The system registered a zillion calls where people talked about fraud world wide, the manager in charge was assigned 20 people to deal with it, and after naturally getting overwhelmed and scapegoated for all kinds of shit, he puts in a request for few hundred more, saying he really needed thousands of people. Corporate wonderland gives him another 20 and writes a para in their annual report about how they are at the forefront of combatting fraud etc etc.
This is how the world works. The chimp troupe hallucinates across the board, at the top and at the bottom about what is really going on. Why?
Because that 3 inch chimp brain has hard limits to how much info, complexity and unpredictability it can handle.
Anything beyond that, the reaction is similar to ants running around pretending they are doing something useful anytime the universe pokes the ant hill.
Herbert Simon won a nobel prize for telling us we don't have to run around like ants and bite everything anytime we are faced with things we can't control.
Godzilla cats really seems like it needs a movie.
Talking to an LLM like a human is like talking to a mirror. You're just shaping their responses based on what you say. Quite sad to see stuff like the "myboyfriendisai" reddit