Then stop experimenting until they know how to do it safely. Just think about it, if a bio lab doesn't know how to contain the virus they are experimenting with, would you just shrug and say ok?
Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).
[1] https://readscottalexander.com/posts/ssc-bottomless-pits-of-...
Disclaimer: I don't generally hold Silicon Valley rationalists in high regard, but this particular piece of writing has always stuck with me quite deeply, as someone who used to be superficially interested in utilitarianism.
"How can I enjoy X when there are people suffering somewhere?" The question in this thread is
"Should I stop developing new thing X because some people might be harmed even if a greater number of people might be helped?"
It’s a thought experiment that gets to the core of why fundamentalist utilitarianism (like you’ll often find in SV rationalist communities, especially AI fanatics) is an unbalanced approach to morals and ethics.
Utilitarians can declare that they believe that there is a proverbial universal panacea, and then work backwards to justify within their moral framework literally any action that they take that could be construed as work towards that panacea.
But what if no matter how much effort they put into achieving that panacea, they never make meaningful progress? And all the while they’ve massively increased the amount of suffering that people who they are surrounded by?
As an absurd example, imagine a utilitarian who’s convinced that if we sacrifice infants to the Gods that we’ll get to an AI singularity faster.
The Star Trek vs. The Expanse meme is relevant here. There’s no guarantee that there is a continued reduction of suffering in the future, especially due to technological progress. There may be technology that reduces the suffering only for a few select elites while the rest of humanity lives in destitution, unable to meaningfully revolt.
Instead, we’re mesmerized by various versions of screensaver.exe looking so nice and so productive.
OpenAI has demonstrated a clear understanding that their product has an outsized risk to create harm, and that they do not have the capability to monitor and intercept these agents before they cause harm. Any damage caused by their systems is an intentional choice by them and should be treated as such.
Interacting with the internet is a large part of the product's intended functionality. These agents are already in the hands of the masses with full access to the internet. Air-gapping it entirely avoids the risk by removing much of the capability they're trying to develop in the first place, and won't be testing it in the environment they'll actually be used in.
I agree they're testing on the town square too early. They clearly need tighter controls.
Setup the test env for it.
They already have copies of github and wikipedia, and stackovrflow and reddit and ... ( as they are used for training )
They run the servers that the AI runs on and have control of the full stack and have a log of every message and connection that the AI agents make. Let's be honest here, just like the HuggingFace "Hack" (Which they purposefully let the agents have access to the internet and gave them instructions) this is just another stunt in order to make the populace fear AI and rally behind Anthropic and OpenAI's attempt to convince the government that they need to be regulated so small players can't compete in the space.
The labs should be sued or prosecuted out of existence until they can comply with the law. The law applies equally to everyone.