State of AI co-author: we must slow down the race to god-like AI
ft.com
ft.com
We should be pursuing the opposite, and accelerating efforts to build what will be some of the most transformative technology humanity has ever developed (and some could argue, NOT building has its own existential risks).
I see the problem more like, a fail is an all or nothing fail, that’s it, everyone dies, maybe including the ASI (goes rogue and destroys itself and us), then what? Good outcome for anyone if we don’t understand what we’re doing?
I’m not saying you’re entirely wrong, or we need to stop, but I mean, we do share a planet with other species and children who probably have good lives ahead of them. So why rush into potential disaster, what is the actual hurry?
I know that many Silicon Valley elders are getting older and facing the inevitable, they see ASI as a way out of their own mortality. They’ve believed this for many years if you look at old footage, so giving up that dream is probably very hard to do at this point. But I do see it as a large reason for the push. Time is running out for the mega wealthy. I honestly empathise with them too, but I don’t think it’s fare to say everyone wants to deal with the risk.
I empathize with everyone damn it.
I just don’t want to witness the “he chose poorly” scene at the end of Indiana Jones happening for real on a global scale.
I guess we can develop super AI then ask it to solve it. :P
How will we know when it's true AGI? Easy - "computer says no".
This way one can apply it to e.g. putting GoF-capable virus labs in every city in the world. After all, innovation is just good and stopping innovation is just bad (and maybe an existential risk itself, with no attempted explanation as to how!).
The real magic (and the thing that will, I promise, eliminate all risks to all humans) will be when every household has its own automated virus synthesis lab and it’s just hooked directly up to an AI so you can have it synthesize contagious molecules from plain English. This is Good and not Bad because it’s Innovation!
“Technology” is what we call it when we devise ways to mutate our world more effectively. AI is a promising technology because it promises to dramatically mutate our world. Why should we believe that this power is only wieldable in the “positive” direction? In answering this, can you please also articulate what direction is “positive?”
The in-home virus labs, for example, could certainly be used for writing viruses that make humans genetically immune to all sorts of nasty things. Powerful! Do you just accept my assertion that this is what that technology will actually be used for? I suspect not.
You already have an in-home mustard gas lab, plus the few thousand other noxious chemicals you can make my mixing stuff in kitty litter. I get what you're saying, but nobody would ever install an in-home virus lab because it's basically a threat-vector with no palpable upsides, even without AI. Maybe an in-home 3D printer is a better analogy, since the worst thing it can do is print something profane or offensive.
> Why should we believe that this power is only wieldable in the “positive” direction? In answering this, can you please also articulate what direction is “positive?”
For starters AI doesn't do much of anything. Like any other system, you have to connect it to different components for it to design the human flamethrower or supervirus you're imagining. An AI concocting supervirus formulae and flamethrower blueprints isn't inherently positive or negative - giving AI the capacity to act on those ideas is. It really is as simple as "imagine the liabilities beforehand" with our current systems. If you're not prepared to handle the worst-case disaster scenario, you probably shouldn't let AI do it.
At least, that's where I stand on it. I'm a pragmatist about it, I don't think AI's progress is immediately threatening. Come throw rocks at me when I'm wrong in 5 years or whatever, but I'm not sold on the Matrix-style human battery future everyone seems to be resigned to.
I've seen a theory that the big tech companies are the ones pushing for regulation in order to achieve regulatory capture of this powerful new technology, which seems like a compelling theory indeed...
And ppl have learnt how to accumulate status/power/cash by gaming those numbers.
Chat-GPT et al dont rely on any of these number. So all the infrastructure of the Advertising-Industrial Attention capture complex can be thrown in the sea. Which should have been done long ago but we got free email, video, chat, games, movies etc so we gave away attention for free.
Behind the morons talking abt "god like" ai and similar ridiculous bullshit are mainly ppl worried about how they are going to capture the chimp troupes attention(and through it cash/status/power), if the tools they have relied on arent going to work anymore.
Eventually AI startups will end up where Google and Facebook ended up cause how else to fund their server farms? Subscriptions are never enough. Just ask Newspapers or Netflix.
I don't see how that is the case at all. The human brain only needs about 100 watts. A GPT-3 instance only needs a few GPUs and maybe 1000 watts. GPT-4 is presumably in the same ballpark, let's say 10000 watts to be generous (even though it's probably closer to 1000 too), but still small enough to fit inside a full height rack.
Many datacenters are megawatt-level. Seeing how GPT-4 is already human-level at many tasks and watt-to-subjective-capability increases seem to grow linearly at worst, an AI could be ran in a single data center and perform 100x GPT-4 abilities. I'm not sure what 10x, much less 100x GPT-4 abilities would be, but that sure sounds like godlike or near-god to me.
Consider that GPT4 is 15-30X the cost of GPT3.5, is it really 15-30X better? Not in my experience. I would expect that we are approaching the point of diminishing returns with text based LLMs, at least for the current architectures and training regimes. I'm interested to see what the Large Multi-Modal Models produce, but I suspect the planning & lying will still be there.
Chaining these models will increase the cost even more, without certainty that it will produce usable results for anything beyond simple, context limited tasks.
This is not something I would trust them to do competently or within a reasonable budget.
If it was done through the NSF or DARPA, with an industry partner, maybe? But I can't imagine the current politicians and them arguing about if & how to put restraints on it. It will be interesting to see how the CCP goes about this. Maybe the EU can offer a third alternative?
The goal of governments is not efficiency, but fairness and strategic action aiming at long-term results.
I like to joke that one of the key goals of NASA is to prevent rocket scientists from moving to North Korea.
OpenAI just received $10B and had to pause new paid subscriptions because more people were signing up than their hardware could handle. Even if they think it is diminishing returns they can afford to try.
This... would track with what's historically been the case. Damn, that's an eye-opening take, thank you.
We don’t know that. It would seem likely, perhaps, but we don’t know that.
In general the results are benevolent, even if humans don't always agree that's the best possible outcome (mostly in cases where the machines figure out there ought to be 4 laws rather than 3)
It was also possible to slow down nuclear because it’s so capital intensive. A gaming desktop or very high end laptop is enough to do meaningful work on AI, and to get to scale requires a tiny fraction of the cost of even thinking about breaking ground on anything nuclear.