And put a stop to LLM progress? Horrible move. I’d rather take the minimal losses/breaches and not encumber the greatest thing discovered over the last few decades with regulations.
Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).
[1] https://readscottalexander.com/posts/ssc-bottomless-pits-of-...
Disclaimer: I don't generally hold Silicon Valley rationalists in high regard, but this particular piece of writing has always stuck with me quite deeply, as someone who used to be superficially interested in utilitarianism.
The Star Trek vs. The Expanse meme is relevant here. There’s no guarantee that there is a continued reduction of suffering in the future, especially due to technological progress. There may be technology that reduces the suffering only for a few select elites while the rest of humanity lives in destitution, unable to meaningfully revolt.
"How can I enjoy X when there are people suffering somewhere?" The question in this thread is
"Should I stop developing new thing X because some people might be harmed even if a greater number of people might be helped?"
It’s a thought experiment that gets to the core of why fundamentalist utilitarianism (like you’ll often find in SV rationalist communities, especially AI fanatics) is an unbalanced approach to morals and ethics.
Utilitarians can declare that they believe that there is a proverbial universal panacea, and then work backwards to justify within their moral framework literally any action that they take that could be construed as work towards that panacea.
But what if no matter how much effort they put into achieving that panacea, they never make meaningful progress? And all the while they’ve massively increased the amount of suffering that people who they are surrounded by?
As an absurd example, imagine a utilitarian who’s convinced that if we sacrifice infants to the Gods that we’ll get to an AI singularity faster.