> We don’t expect Adobe to restrict the content that can be created in Photoshop. We don’t expect Microsoft to have acceptable use policies for what you can write in Microsoft Office.

Photoshop and Office don't (yet) conjure up suicide lullabys or child nudity from a simple user prompt or button click. If they did, I would absolutely expect to hold them accountable.

The political economy equilibrium enabled by technology very much goes the other way though. Once politicians realize they can surveil everyone in real time for wrongthink and wrongspeak they have existential incentives to seize that power as fast as possible, lest another power center seize it instead and use it against them. That is why you are seeing the rise of totalitarianism and democratic backsliding everywhere, because the toxic combination of asymmetric cryptography (for secure boot/attestation/restricting what software can run), always online computers, and cheap data processing and storage leads to inexorable centralization of soft and hard power.

Some manufactures of knives could still be recalled for safety reasons, and MS Office/Google Drive certainly have content prohibitions in their TOS once you’re dealing with their online storage. I agree with your metaphor in that I doubt much use would come from banning AI entirely, but I feel there must be some viable middle ground of useful regulation here.

"We shouldn’t outlaw kitchen knives because people can cut themselves."

How about if the knife would convince you to cut yourself?

If you encourage someone to kill themselves, you are culpable. OpenAI should meet that standard too.

It's currently in the sixth page