Jensen Huang does not want regulation, but in his interview with Ezra Klein he said AI will let us “know anything” and “do anything”. Do we actually want any random person to be able to “do anything”? I remember when the Japanese doomsday cult Aum Shinrikyo attacked the Tokyo subway with sarin. Do we want doomsday cults to be able to “know anything” and “do anything” so that instead of releasing sarin, they release a genetically modified strain of smallpox?
This is not a hypothetical risk, this is an actual, present danger. And good luck trying to vaccinate yourself against an engineered superflu using a Chinese open weight model.