Let’s say you deploy it in production and a user comes back and says “Why is this prompt considered harmful?”
You have no way to provide a concrete reason to the user at that point.
2016 scenario: An end user contacts the company, and a customer service rep answers the ticket, saying they’re sorry and explaining that they’ve sent the feedback to the team, and the team may even receive at least a summary of complaints received about the system.
2026 scenario: all contact information has been scrubbed from the site. Users can click “chat” and a chatbot will apologize for their dissatisfaction and offer no option to escalate. No one will ever hear anything about the complaint, so there’s no need to explain the failure. User can either accept this or can get f**ked because all competitors operate the same way.
There are also new rules about AI and what it can be used for. Mostly this restricts the government from AI-enhanced surveillance, which is good. But there are also issues regarding job security and automatically categorizing people based on AI.
So this is super useful, but has potential issues depending on how it is deployed.
You should just be able to look at the user's message and tell them why it's against your policy, else reverse the decision if you see no violation.
If a user is curious specifically about how the model made its decision, and you want to reveal detail at that level, it's an open-weights model so interpretability techniques should work ("biggest impact on score came when focusing on this word in your message and this part of the policy").
The reality is that most users don’t ask because they know they violated the rule.