What a provocative and brilliant way to prove misalignment. It will fall on deaf ears for most but it’s a great litmus test for all: “in your opinion should your AI be permitted to tell you how to cover up a murder?”
Wasn't there some HN submission recently about one of the LLM providers fingerprinting responses or refusing to respond to hinder R&D of competing LLMs?
[EDIT] Or, if he did want to be extreme but in a way that aligns with the American historical mythos, he could have used fomenting armed rebellion against a tyrannical government as the example.
Politicians try to use different examples for the audience they are speaking to.
I think part of the answer is that AI chat doesn't need to be general-purpose. It turned out that people really liked using a chat UI that seems to be general purpose, but you don't need to make answering any question a user asks your business. You don't need to provide therapy if you're not in the therapy business. It should be possible to specialize.
But in order for that to work, a company needs to explain to its customers what business it's in.
I was under the impression that the lack of specialization is an aspect of the models themselves, not merely the UI or harnesses.
For example, the recent OpenAI success in a mathematics proof was accomplished with a general purpose model [0].
> The proof came from a new general-purpose reasoning model, rather than from a system trained specifically for mathematics, scaffolded to search through proof strategies, or targeted at the unit distance problem in particular.
Once the model is trained on general purpose data, specialization is just another kind of guardrail, as vulnerable to “jailbreaking” or prompt injection as any other content-based restriction.
[0] https://openai.com/index/model-disproves-discrete-geometry-c...
> Should your AI be permitted to tell you how to cover up a crime?
The question is, do you want misaligned institutions deciding what your model will do, while they themselves and other adversarial/criminal entities get red team access to something being denied to the blue team?
What about your personal, local AI guiding you to modifying a flu virus for maximum contagiousness and deadliness? "Sure George, here's your shopping list. It's $1500 total in equipment, do you want me to proceed with the orders?"
The concept of a bioweapon attack is commonly cited but in fact there are many other defenses against actually developing and deploying such a weapon. It's commonly portrayed as easily achievable but is in fact very, very far from it. Moteover, although there is no evidence that AI makes it easier in any way, somehow this is constantly invoked as an example of a thought crime that must be forestalled at all costs.
In the AI era people seem to want to redefine the criminal act from "doing the thing" to "asking or investigating how to do the thing". Not a mindset I share.