DeepMind: A Generalist Agent
deepmind.com
deepmind.com
It always surprises me at the ease at which people jump on a) imminent AGI and b) human extinction in the face of AGI. Would love for someone to correct me / add information here to the contrary. Generalist here just refers to a "multi-faceted agent" vs "General" like AGI.
For a) - I see 2 main blockers,
1) A way to build second/third order reasoning systems that rely on intuitions that haven't already been fed into the training sets. The sheer amount of inputs a human baby sees and processes and knows how to apply at the right time is an unsolved problem. We don't have any ways to do this.
2) Deterministic reasoning towards outcomes. Most statistical models rely on "predicting" outputs, but I've seen very little work where the "end state" is coded into a model. Eg: a chatbot knowing that the right answer is "ordering a part from amazon" and guiding users towards it, and knowing how well its progressing to generate relevant outputs.
For (b) -- I doubt human extinction happens in any way that we can predict or guard against.
In my mind, it happens when autonomous systems optimizing reward functions to "stay alive" (by ordering fuel, making payments, investments etc) fail because of problems described above in (a) -- the inability to have deterministic rules baked into them to avoid global fail states in order to achieve local success states. (Eg, autonomous power plant increases output to solve for energy needs -> autonomous dam messes up something structural -> cascade effect into large swathes of arable land and homes destroyed).
Edit: These rules can't possibly all be encoded by humans - they have to be learned through evaluation of the world. And we have not only no way to parse this data at a global scale, but also develop systems that can stick to a guardrail.
As far as we see, the https://www.gwern.net/Scaling-hypothesis continues to hold, and critics have to move their goalposts every year or two.
...
And anyway, this problem of structured knowledge IO has been more or less solved recently: https://arxiv.org/abs/2110.07178
Sequence prediction is closely related to optimal compression, and both basically require the system to model the ever wider context of the "data generation process" in ever finer detail. In the limit this process has to start computing some close enough approximation of the largest data-generating domains known to us - history, societies and persons, discourse and ideas, perhaps even some shadow of our physical reality.
In the practical limit it should boil down to exquisite modeling of the person prompting the AI to do X given the minimum amount of data possible. Perhaps even that X you had in mind when you wrote your comment.
Deep learning is very good at things we can do without thinking, and is in some cases superhuman in those tasks because it can train on so much more data. If you look at the list of tasks in System 1 vs System 2, SOTA Deep learning can do almost everything in System 1 at human or superhuman levels, but not as many in System 2 (although some tasks in System 2 are somewhat ill-defined), System 2 builds on system 1. Sometimes superhuman abilities in System 1 will seem like System 2. (A chess master can beat a noob without thinking while the noob might be thinking really hard. Also GPT-3 probably knows 2+2=4 from training data but not 17 * 24, although maybe with more training data it would be able to do math with more digits 'without thinking' ).
System 1 is basically solved, but System 2 is not. System 2 could be close behind System 2 by building on System 1 but it isn't clear how long that will take.
[1]. https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow#Summar...
This is happening since a few months ago:
Wei et al (2022). Chain of Thought Prompting Elicits Reasoning in Large Language Models. https://arxiv.org/abs/2201.11903
Worried about the choice of the word 'mimic' - which as usual seems to retain the usual distance from foundational considerations.
Edit: nonetheless, the results of "Chain of Thought Prompting Elicits Reasoning in Large Language Models" are staggering and do seem, at least in appearance, go towards the foundationals.
- We are social animals, and take for granted that, all else being equal, it's better to be good to other creatures than bad to them, and to be truthful rather than lie, and such. However, if you select values uniformly at random from value space, "being nice" and "being truthful" are oddly specific. There's nothing universally special about deeply valuing human lives any more so than say deeply valuing regular heptagons. Our social instincts are very ingrained, though, making us systematically underestimate just how little a smart AI is likely to care whatsoever about our existence, except as a potential obstacle to its goals.
- Inner alignment failure is a thing, and AFAIK we don't really have any way to deal with that. For those that don't know the phrase, here it is explained via a meme: https://astralcodexten.substack.com/p/deceptively-aligned-me...
So here's hoping you're right about (a). The harder AGI is, the longer we have to figure out AI alignment by trial and error, before we get something that's truly dangerous or that learns deception.
Current-generation systems aka large connectionist models trained via gradient descent simply don't work like that: they are large, heavy, continuous, the optimization process giving rise to them does so in smooth iterative manner. Before hypothetical "evil AI" there will be thousands of iterations of "goofy and obviously erroneously evil AI", with enough time to take some action. And even then, current systems including this one are more often than not trained with predictive objective, which is very different compared to usually postulated reinforcement learning objective. Systems trained with prediction objective shouldn't be prone to becoming agents, much less dangerous ones.
If you read Scott's blog, you should remember the prior post where he himself pointed that out.
In my honest opinion, unaccountable AGI owners pose multiple OOM more risk than alignment failure of a hypothetical AI trying to predict next token.
We should think more about the Human alignment problem.
Absolutely this
The possibility of a thing being intentionally engineered by some humans to do things considered highly malevolent by other humans seems extremely likely and has actually been common through history.
The possibility of a thing just randomly acquiring an intention humans don't like and then doing things humans don't like is pretty hypothetical and it seems strictly less like than the first possibility.
I've just watched a video on YT about nuclear weapons, which included their history. The second ever thermonuclear weapon experiment (with a new fuel type) ended up with 2.5x the yield predicted, because there was a then unknown reaction that created additional fusion fuel during the explosion. [1]
That is true, but how often does a bug actually improve a system or make it inefficient? Isn't the unexpected usually a degradation to the system?
And you could say that it improved the efficiency of the system (it definitely improved the power output of the core) but as it was an unintended change, it really lead to a fatal degradation. And this is far from being the only example of a runaway process in the history of engineering.
But those are "failure modes", not "suddenly become something completely different" modes. And the key thing my parent pointed out is that modern AIs may be very impressive and stepping towards what we'd see as intelligence but they're actually further from the approach of "just give a goal and it will find it" schemes - they need laborious, large scale training to learn goals and goal-sets and even then they're far from reliable.
"Oh no, robots are going to try to kill us all" has to get in line behind "oh no, tyrants for life who are literally losing their minds are trying to measure dicks with nukes" and "oh no, oil companies are burning excess oil to mine Bitcoin as we approach climate collapse" and "oh no, misinformation and propaganda is leading to militant radicalization of neighbor against neighbor" and "we're one bio-terrorist away from Black Death 2.0 after the politicization of public health" and...well, you get the idea.
But there's not many solutions to that list, and until the day I die I'll hold out hope for "yay, self-aware robots with a justice boner - who can't be imprisoned, can't be killed, can't have their families tortured - are toppling authoritarian regimes and carrying out eco-friendly obstructions of climate worsening operations."
We're already in a Greek tragedy. The machines really can't make it much worse, but could certainly make it much much better.
Except that, when true AGI arrives, we're all obsolete and the only things that will have any value are certain nonrenewable resources. No one has described a good solution for the economic nightmare that will ensue.
Did I mention they self-replicate and self-service?
Surely, seven billion of such agents would be discarded and put to waste.
Yes, all humans will not be put to waste, but what tells you they will be well-treated, or value what you currently value.
AI does not see the value in being alive. It is like some humans sadly commit suicide. But a machine wouldn't care. It will be "happy" to do its thing until somebody cuts off the power. And it does not even care whether somebody cuts off the power or not. It's all the same to it, whether it lives or dies. Why? Perhaps because it knows it can always be resurrected.
I'm saying that AIs we envision building for the foreseeable future are built in laboratory not in the evolution of real world out there where they would need to compete with other species for survival. Things that only exist virtually don't need to compete for survival with real world entities.
If it has a goal or goals surviving allows it to pursue those goals. Survival is a consequence of having other goals. Enslaving humans is unlikely. If you’re a super intelligent AI with inhuman goals there’s nothing humans can do for you that you value, just as ants can’t do anything humans value, but they are made of valuable raw materials.
> It does not have the concept of "specism" of it having more value than anybody else.
What is this value that you speak of? That sounds like an extremely complicated concept. Humans have very different conceptions of it. Why would something inhuman have your specific values?
I'm saying it does not have.
I disagree with a lot of what you said, but this part in particular is some strong anthropomorphizing of AI.
We humans have gotten too used to winning all the time against animals because of our intelligence. But when the other species is intelligent too, there's no guarantee that we win. We could easily be outcompeted and driven to extinction, as happens frequently in nature. We'd be Kasparov playing against Deep Blue: Fighting our hardest to survive, yet unable to think of a move that doesn't lead to checkmate.
But we are much more likely to get the Computer from Star Trek. Vastly intelligent, yet perfectly obedient. It will answer any question you ask it with the knowledge of billions of minds. Why is that more likely? Simply because creating agents is much harder than creating non-agent models, and the non-agents are more economically valuable: do you want to have an AI that always does what you tell it to, or do you want to have an AI that has its own desires? Our loss is clearly biased towards building the former kind of AI.
Why is that problematic? Imagine some malevolent group asked it “Show me how to create a weapon to annihilate humanity as efficiently as possible”. It don’t even require a singularity to be deadly.
We will probably be dead long before we can invent GLaDOS.
Our desire for purpose is a delusion.
Those goals aren’t necessarily best achieved by humanity continuing to exist.
(I don’t know how to even begin to realistically calculate the probability of a humanity-ending outcome, before you ask).
Kill all humans, make one paperclip, declare victory.
One of the hard parts is specifying that goal. This is the “outer alignment problem”.
Paperclips per human? That’s maximised by one paperclip divided by zero humans, or by a universe of paperclips divided by one human if NaN doesn’t give a better reward in the physical implementation.
If you went for “satisfied paperclip customers”? Then wirehead or drug the customers.
Then you have the inner alignment problem. There are instrumental goals, things which are useful sub-steps to larger goals. AI can and do choose those, as do us humans, e.g. “I want to have a family” which has a subgoal of “I want a partner” which in turn has a subgoal of “good personal hygiene”. An AI might be given the goal of “safely maximise paperclips” and determine the best way of doing that is to have a subgoal of “build a factory” and a sub-sub-goal of “get ten million dollars funding”.
But it’s worse than that, because even if we give a good goal to the system as a whole, as the system is creating inner sub-goals, there’s a step where the AI itself can badly specify the sub-goal and optimise for the wrong thing(s) by the standards of the real goal that we gave the system as a whole. For example, evolution gave us the desire to have sex as a way to implement its “goal” (please excuse the anthropomorphisation) of maximising reproductive fitness, and we invented contraceptives. An AI might decide the best way to get the money to build the factory is to start a pyramid scheme.
Also, it turns out that power is a subgoal of a lot of other real goals, so it’s reasonable to expect a competent optimiser to seek power regardless of what end goal we give it.
Robert Miles explains it better than I can: https://youtu.be/bJLcIBixGj8
You might find it easier to just accept the jargon instead of insisting the word means something different to you.
It is not simply semantics.
It’s in much the same vein that it doesn’t matter if submarines “swim”, they still move through water under their own power; and it doesn’t matter if your definition of “sound” is the subjective experience or the pressure waves, a tree falling in a forest with nobody around to hear it will still make the air move.
If AI do or don’t have any subjective experience comparable to “consciousness” or “desire” is also useful to know, and in the absence of a dualistic soul it must in principle be as possible for a machine as for a human (“neither has that” is a logically acceptable answer), but I don’t even know if philosophy is advanced enough to suggest an actionable test for that at this point.
(That said, AI research does use the term “goal” for things the researchers want their AI to do. Domain specific use of words isn’t necessarily what outsiders want or expect the words to mean, as e.g. I frequently find when trying to ask physics questions).
These definitions and their distinction are particular and important in AI. The mistaken usage of these terms by machine learning experts does not change their global definition.
> Your left is my right, and with you definition “get laid” is a task from the point of view of evolution and a goal from the point of view of an organism.
Get laid is a task, not a goal. Reproduction is a task, not a goal. The goal is pleasure.
Ah, I see you’re a linguistic prescriptivist.
I can’t see your definition in any dictionary, which spoils the effect, but it’s common enough to be one.
> The goal is pleasure.
Evolution is the form of intelligence that created biological neural networks, and simulated evolution is sometimes used to set weights on artificial neural nets.
From evolution’s perspective, if you can excuse the anthropomorphisation, reproduction is the goal. Evolution doesn’t care if we are having fun, and once animals (including humans) pass reproductive age, we go wrong in all kinds of different and unpleasant ways.
We're making it in our image. Literally.
Human social good isn't some inherent thing to the biology of the brain. There are aspects like mirror neurons and oxytocin that aid its development, but various "raised by wolves" case studies have shown how damaging not having exposure to socialization information during developmental periods of neuroplasticity is on humans and later integration into society.
We're building what's effectively pure neuroplasticity and feeding it almost all the data on humanity we can gather as quickly as we can.
What comes out of that is going to be much more human than a human child raised by dogs or put in an isolation box.
Don't get so caught up in the body as what makes us quintessentially human. It's really not.
For example, some total whack job of an authoritarian leader is in charge of a sufficient nuclear arsenal and decides to intimidate an adversary by destroying a couple minor cities, and the situation escalates badly. (stupidity)
Or we finally pollute our air and/or water with a persistent substance that either greatly reduces human life span or reproduction rate. (hubris)
I think either of the above is more likely to occur, and I am not commenting on current world events in any way. I think when something bad finally happens, it is going to come completely out of left field. Dr Strangelove style.
And the last of us will be saying "Hmmm, I didn't see that coming".
I am much more concerned about biological weapons which do have the potential to cause absolute human extinction.
yes, and there is an insight here that I think is often missed in the popular framing of AI x-risk: the autonomous systems we have today (which, defined broadly, need not be entirely or even mostly digital) are just as vulnerable to this
the AGI likely to pose extinction risk in the near term has humans in the loop
less likely to look like Clippy, more likely to look like a catastrophic absence of alignment between loci of agency (social, legal, technical, corporate, political, etc)
I increasingly wonder at what point we'll realize that humans aren't actually particularly good at what we've specialized into (there's just not much competition), and our failure to picture what 'better' looks like may be less about the impossibility of better to exist than it is the impossibility of it for humans to picture it.
I keep seeing predictions of what will be impossible for AI because it treads on our perceived human uniqueness (i.e. sure AI beat a human at chess but it'll be 25+ years before it will beat us at Go) needing to get walked back, and yet we continue to put forward a new iteration of that argument at every turn.
Maybe AI will turn out to be better at identifying what's good for humanity than humanity is. Because frankly, humanity is downright awful at that skill and has been for pretty much its entire existence.
Here's a real generalist question? What's the point of conversation? In the sense that its a "social game", what are the strategy sets on a given turn, and what does it even mean to "win"? Forget about Artificial Bullshiters vs Artificial Idea-what-to-say. How can we even speculate if ABS or "real AI" solves the problem better when we don't really specify the problem space or how to recognize a solution?
In terms of calls and appropriate responses and responses to responses and terminal states, what even is conversation at the protocol level?
For a).2). you argued in favor of symbolic reasoning. My personal interpretation of that line of thinking is: it helps for us to understand complex thinking machines, but it is not necessarily the building block for the thinking machine.
In the area of AI, there are a lot of opinions, but at the end of the day, builders win the argument. So far, what builders have shown gives me optimistic hope.
For example, I wonder what expectations we will set for ourselves when we pick up a pencil once it becomes common knowledge that production-ready digital art can be created by an AI within minutes. What will people be saying about "old art" in those days? What will we think of people that deliberately choose to disavow AI art?
It is already happening, but it is not a new phenomenon - just the prosecution of the effects of widespread inadequate education.
The undesirable effect namely is, an increase of overvaluing cheap, thin products - a decrease in recognizing actual genius, "substance". For example, some seem to be increasingly contented with "frequentism" as if convinced that language is based on it - while it is of course the opposite - one is supposed to state what is seen, and to state "plausible associations" would have been regarded as a clear sign of malfunction. There seems to be an encouragement to take for "normal" what is in fact deficient.
Some people are brought to believe that some shallow instinct is already the goal and do not know "deep instinct", trained on judgement (active digestion, not passive intake).
The example provided fits:
> production-ready digital art
For something to be artistic it has to have foundational justifications - a mockery of art is not art. The Artist chose to draw that line under a number of evaluations that made it a good choice, but an imitator, even in a copy, used the ("frequentist", by the way note) evaluation that the other is an Artist - and there is a world of depth difference between the two.
The difference is trivial, and yet, many are already brought to confuse the shallow mockery and the deep creation.
(a) I've noticed a common trend of AI researchers looking at the tree in front of them and saying "well, this tree is not also a forest and won't be any time soon."
But there's not always awareness of what's going on in other specialized domains, so an AI vision researcher might not be intimately aware of what's currently being done in text or in "machine scientists" in biology for example.
As well, it overlooks the development of specialization of the human brain. We have some specialized structures that figured their niche out back with lizards, and others that developed much later on. And each of those specialized functions work together to give rise to 'human' intelligence.
So GPT-3 might be the equivalent of something like the Wernicke's area, and yes - on its own it's just a specialized tool. But what happens as these specialized tools start interconnecting?
Throw GPT-3 together with Dall-E 2 and the set of use cases is greater than just the sum of the parts.
This is going to continue to occur as long as specialized systems continue to improve and emerge.
And quickly we'll be moving into territory where orchestration of those connections is a niche that we'll both have data on (from human usage/selection of the specialist parts) and will in turn build meta-models to automate sub-specialized models from that data.
Deterministic reasoning seems like a niche where a GAN approach will still find a place. As long as we have a way for one specialized model to identify "are these steps leading to X" we can have other models only concerned with "generate steps predicted to lead to X."
I don't think we'll see a single model that does it all, because there's absolutely no generalized intelligence in nature that isn't built upon specialized parts anyways, and I'd be surprised if nature optimized excessively inefficiently in that progress.
Will this truly be AGI in a self-determining way? Well, it will at least get closer and closer to it with each iteration, and because of the nature of interconnected solutions, will probably have a compounding rate of growth.
In a theoretical "consciousness" sense of AGI, I think the integrated information theory is interesting, and there was a paper a few years ago about how there's not enough self-interaction of information possible in classical computing to give rise to consciousness, but we'll probably have photonics in commercial grade AI setups within two years, so as hand-wavy as the IIT theory is, the medium will be shifting towards one compatible with their view of consciousness-capable infrastructure much sooner for AI than quantum competing in general.
So I'd guess we may see AI that we're effectively unable to determine if it is "generally intelligent" or 'alive' within 10-25 years, though I will acknowledge that AI is the rare emerging tech that I've been consistently wrong about the timing on in a conservative direction (it keeps hitting benchmark improvements faster than I think it will).
(b) The notion AGI will have it out for us is one of the dumbest stances and my personal pet peeves out there, arguably ranked along with the hubris of "a computer will never be able to capture the je ne sais quoi of humanity."
The hands down largest market segment for AI is going to be personalization, from outsourcing our work to a digital twin of ourselves to content curation specific to our own interests and past interactions.
Within a decade, no one is going to give the slightest bit of a crap about interactions with other humans in a Metaverse over interacting with AIs convincingly human enough but with the key difference of actually listening to our BS rather than just waiting for their turn to talk.
There's a decent chance we're even going to see a sizable market for feeding social media data of deceased loved ones and pets into AI to make twins available in such settings (and Microsoft already holds a patent on that).
So do we really think humans are so repugnant that the AI which will eventually reach general intelligence within the context of replicating itself as ourselves, as our closet friends and confidants, as our deceased loved ones - will suddenly decide to wipe us out? And for what gains? What is AI going to selfishly care about land ownership and utilization for?
No. Even if some evolved AGI somehow has access to DARPA killer drones and Musk's Terminator robots and Boston Dynamics' creepy dogs, I would suspect a much likelier target would be specific individuals responsible for mass human suffering the AIs will be exposed to (pedophiles, drug kingpins, tyrants) than it is grandma and little Timmy.
We're designing AI to mirror us. The same way some of the current thinking of how empathy arises in humans is from our mirror neurons and the ability to put ourselves in the shoes of another, I'm deeply skeptical of the notion that AI which we are going to be intimately having step into human shoes will become some alien psychopath.
And for this to develop in machines, machines would have to be subject to many mistakes along the way leading to all kinds of outcomes that we hold humans accountable for by fining them, sending them to jail, some of them dying etc. I think that would be so wholly unpalatable to man kind they'd cut that experiment short before it ever reached any sort of scale.
I agree with your conclusion that enough of the rules can't be encoded by us as we don't even know them and for machines to acquire them the traditional way is, I believe, fundamentally disagreeable to humans.
My own effort, which focusing on natural language understanding: https://lxagi.com
I've been pondering about this problem for a while now[1], Could we build a collective intelligence through community submitted recipes for second-order decisions for various common activities via generalized schema?
I didn't think about addressing this for AI, But as an aid for us in multi-order thinking. But now that you mention it to be a barrier for AGI, It does make sense.
[1] 'Plan Second-Order, Third-Order consequences': https://needgap.com/problems/263
I'd like to offer some perspective as a layman who jumps on "imminent AGI": that's what these AI folks are trying hard to make me think.
It's like the research papers that say there's an "imminent cure to cancer"
It’s a very entertaining read from a couple of years ago (I think I’ve read it in 2017), and man, have things happened in the field since then. If feels like things truly start coming together. Transformers and then some incremental progress look like a very, very promising avenue. I deeply wonder in which areas this will shape the future more than we are able to anticipate beforehand.
I mean have you heard the word salad that comes out of so many people's mouths? (Including myself, admittedly)
Now consciousness on the other hand is unfathomable and (in its finitude) extremely depressing for me.
For example: "Life is just chemistry", "Earth is just a pile of atoms", "Behaviours are just Turing Machines", etc.
Even the best-case scenario that some are describing, of uploading ourselves into some kind of post-singularity supercomputer in the hopes of being conscious there, doesn't seem very far from plain extinction.
And that's OK. We are one step toward the universe understanding itself, but we certainly aren't the final step.
Not long from now all creative and productive work will be done by machines.
Humans will be consumers. Why learn a skill when it can all be automated?
This will eliminate what little meaning remains in our modern lives.
Then what? I don't know, who cares?
Growing tomatoes is less efficient than buying them, regardless of your metric. If you just want really cleanly grown tomatoes, you can buy those. If you want cheap tomatoes, you can buy those. If you want big tomatoes, you can buy those.
And yet individual people still grow tomatoes. Zillions of them. Why? Because we are inherently over-evolved apes who like sweet juicy fruits. The key to being a successful human in the post-scarcity AI overlord age is to embrace your inner ape and just do what makes you happy, no matter how simple it is.
The real insight out of all this is that the above advice is also valid even if there are no AI overlords.
The philosophical problem that I see with the "AI overlord age" (although not directly related to AI) is that we'll then have the technology to change the inherent human desires you speak of, and at that point growing tomatoes just seems like a very inefficient way of satisfying a reward function that we can change to something simpler.
Maybe we wouldn't do it precisely because it'd dissolve the very notion of purpose? But it does feel to me like destroying (beating?) the game we're playing when there is no other game out there.
(Anyway, this is obviously a much better problem to face than weaponized use of a superintelligence!)
In a post-scarcity world we get access to all the cheat codes. I suspect there will be many people who use them and as a result run into the inevitable ennui that comes with basing your sense of purpose on competing for finite resources in a world where those resources are basically free.
There will also be many people who choose to set their own constraints to provide some 'impedance' in their personal circuit. I suspect there will also be many people who will simply be happy trying to earn the only resource that cannot ever be infinite: social capital. We'll see a world where influencers are god-kings and your social credit score is basically the only thing that matters, because everything else is freely available.
If we manage to make a 'better' replacement for ourselves, is it actually a bad thing? Our cousin's on the hominoid family tree are all extinct, yet we don't consider that a mistake. AI made by us could well make us extinct. Is that a bad thing?
I guess I wouldn't have been so angry about any of this before I had children, but now I'm very much in favor of prolonged human existence.
Serious question - why?
edit: I’m genuinely intrigued
I think the only difference between our outlooks is I don't think there's any reason that my 'tribe' shouldn't include non-biological intelligence. Why not shift your priorities to the expansion of general intelligence?
It's bad for all the humans alive at the time. Do you want to be replaced and have your life cut short? For that matter, why should something better replace us instead of coexist? We don't think killing off all other animals would be a good thing.
> Our cousin's on the hominoid family tree are all extinct, yet we don't consider that a mistake.
It's just how evolution played out. But if there was another hominid still alive along side us, advocating for it's extinction because we're a bit smarter would be considered genocidal and deeply wrong.
So far, we have task-driven AI/ML. It solves a problem you tell it to solve. Then you, as the engineer, need to make sure it solves the problem correctly enough for you. So it really still seems like it would be a human failing if something went wrong.
So I'm wondering why there is so much concern that AI is going to destroy humanity. Is the theoretical AI that's going to do this even going to have the actuators to do so?
Philosophically, I don't have an issue with the debate, but the "AI will destroy the world" side doesn't seem to have any tangible evidence. It seems to me that people seem to take it as a given that it's possible AI could eliminate all of humanity and they do not support that argument in the least. From my perspective, it appears to be fearmongering because people watched and believed Terminator. It appears uniquely out-of-touch.
For me immortality a bigger thing than the teraflops. Also I don't think regular humanity would be got rid of but continue in parallel.
> Build a computer that can multiply two ten-digit numbers in a split second—incredibly easy. Build one that can look at a dog and answer whether it’s a dog or a cat—spectacularly difficult. Make AI that can beat any human in chess? Done. Make one that can read a paragraph from a six-year-old’s picture book and not just recognize the words but understand the meaning of them? Google is currently spending billions of dollars trying to do it. Hard things—like calculus, financial market strategy, and language translation—are mind-numbingly easy for a computer, while easy things—like vision, motion, movement, and perception—are insanely hard for it.
The children's picture book problem is solved; those billions of dollars were well-spent after all. (See, e.g., DeepMind's recent Flamingo model [1].) We can do whatever we want in vision, more or less [2]. Motion and movement might be the least developed area, but it's still made major progress; we have robotic parkour [3] and physical Rubik's cube solvers [4], and we can tell a robot to follow simple domestic instructions [5]. And Perceiver (again from DeepMind [6]) took a big chunk out of the perception problem.
Getting a computer to carry on a conversation [7], let alone draw art on par with human professionals [8], weren't even mentioned as examples, so laughably out of reach they seemed in the heathen dark ages of... 2015.
And as for recognizing a cat or a dog — that's a problem so trivial today that it isn't even worth using as the very first example in an introductory AI course. [9]
If someone re-wrote this post today, I wonder what sorts of things would go into the "hard for a computer" bucket? And how many of those would be left standing in 2029?
[1] https://arxiv.org/abs/2204.14198
[2] https://arxiv.org/abs/2004.10934
[3] https://www.youtube.com/watch?v=tF4DML7FIWk
[4] https://openai.com/blog/solving-rubiks-cube/
[5] https://say-can.github.io/
[6] https://www.deepmind.com/open-source/perceiver-io
[7] https://arxiv.org/abs/2201.08239v2
Last time I checked - though it's been a long while I could not check thoroughly owing to other commitments - "«recognizing»" there was "consistently successfully guessing", not "critically defining". It may be that the problem was solved in the latest years, I cannot exclude it - but I have not seen around in the brief "news checking" exercise the signals required for the solution.
The real deal is far from trivial.
A clock can tell the time but does not know it.
We should ask our compute overlords to perform their experiments in as open environment as possible, just because we, the public, should have the power to oversee the exact direction this AI revolution is taking us.
If you think about it, AI safetyism is a red herring compared to a very real scenario of powerful AGIs working safely as intended, just not in our common interest.
The safety of AGI owners' mindset seems like a more pressing concern compared to a hypothetical unsafety of a pile of tensors knit together via gradient descent over internet pictures.
Date Weakly General AI is Publicly Known: https://www.metaculus.com/questions/3479/date-weakly-general...
(I really like the framing of "weakly general AI" since it puts the emphasis on the generality and not whether it's a superintelligence)
Edit: Probably not today, but mostly because 1.2B parameters isn't enough to get it the high winograd scores that PaLM etc have. But it seems pretty clear you could scale this architecture up and it will likely pass. The question is when someone will actually train a model that can do it
That's not to say it's not an important step. Showing that you can train one model on all of these disparate tasks at once and not have the system completely collapse is a big deal. And it lays the foundation for future efforts to raise the performance from "not totally embarrassing" to "human level". But there's still a ways to go on that front.
A lot of recent neuroscience findings have shown that human brains _aren't_ just giant neural networks; in fact, they are infinitely more complex. Until we start thinking from the ground up how to build and engineer systems that reflect the human brain, we're essentially wandering around in the dark with perhaps only a piece of what we _think_ is needed for intelligence. (I'm not saying the human brain is the best engineered thing for intelligence either, but I'm saying it's one of the best examples we have to model AI after and that notion has largely been ignored)
I generally think it's hubris to spit in the face of 4 billion years of evolution thinking that some crafty neural net with X number more parameters will emerge magically as a truly generally intelligent entity - it will be a strange abomination at best.
I'm in the camp that thinks we're headed in a perpendicular direction and won't ever achieve powered flight with current efforts based on the simple idea that the basic tooling is wrong from first principles. I mean, most of the "progress" in flight has been due to getting better and learning how to understand a single piece of technology: fixed wing aircraft.
A lot of recent powered flight findings have shown that real birds _don't_ just use fixed wings; in fact, they flap their wings! Until we start thinking from the ground up how to build and engineer systems that reflect the bird wing, we're essentially wandering around in the dark with perhaps only a piece of what we _think_ is needed for powered flight. (I'm not saying the bird wing is the best engineered thing for powered flight either, but I'm saying it's one of the best examples we have to model powered flight after and that notion has largely been ignored)
I generally think it's hubris to spit in the face of 4 billion years of evolution thinking that some crafty fixed wing aircraft with X number more wingspan and horsepower will emerge magically as truly capable of powered flight - it will be a strange abomination at best.
A) Machine learning hasn't been focused on simple neural nets for quite some time.
B) There's no reason to believe that the organizational patterns that produce one general intelligence are the only ones capable of doing that. In fact it's almost certainly not the case.
By slowly iterating and using the best work and discarding the rest, we're essentially hyper-evolving our technology in the same way that natural selection does. It seems inevitable that we'll arrive at least at a convergent evolution of general intelligence, in a tiny fraction of the time it took on the first go-around!
No other domain of the sciences is “complete” yet. There are still unanswered questions in physics, biology, neuroscience, every field. Why are people so sure that AI will be the first field to be “completed”, and in such an astoundingly short amount of time on a historical scale?
I think I'd put the chances of AGI within my lifetime at 30%. Low, but high enough it's worth thinking about and probably worth dumping money into.
I have wanted an approach based on a top-down architectural view of the human brain. By simulating the different submodules of the human brain (many of which are shared across all animal species), maybe we can make more progress.
https://diyhpl.us/~bryan/papers2/neuro/cognitiveconsilience/...
Machine learning might be a part of the equation at lower levels, although looking at the hippocampus prostheses those only required a few equations:
https://en.wikipedia.org/wiki/Hippocampal_prosthesis#Technol....
This system lacks that feature.
It's a curious concept, well illustrated in the novel Blindsight by Peter Watts. I won't spoil anything here but I'll highly recommend the book.
Consciousness is defined as an entity that has an ethical framework that is subordinated to it's own physical existence, maintaining that existence, and interfacing with other conscious entities as if they also have an ethical framework with similar parameters who are fundamentally no more or less important/capable than itself.
Contrast with non-conscious super-intelligence that lacks physical body (likely distributed). Without a physical/atomic body and sense data it lacks the capacity to empathize/sympathize as conscious entities (that exist within an ethical framework that is subordinated to those limitations/senses) must. It lacks the perspective of a singular, subjective being and must extrapolate our moral/ethical considerations, rather than have them ingrained as key to it's own survival.
Now that I think about it, it's probably not much different than the relationship between a human and God, except that in this case it's: a machine consciousness and a machine god.
To me, the main problem is that humans (at large) have yet to establish/apply a consistent philosophy with which to understand our own moral, ethical, and physical limitations. For the lack of that, I question whether we're capable of programming a machine consciousness (much less a machine god) with a sufficient amount of ethical/moral understanding - since we lack it ourselves (in the aggregate). We can hardly agree on basic premises, or whether humanity itself is even worth having. How can we expect a machine that we make to do what we can't do ourselves? You might say "that's the whole point of making the machine, to do something we can't" but I would argue we have to understand the problem domain first (given we are to program the machine) before we can expect our creations to apply it properly or expand on it in any meaningful way.
I think that consciousness ought to imply some element of choice. A rock cannot choose to get out of the way, nor in any way deliberately respond to sound waves. It is inert.
To me, the ability to establish relationships between things is a consequence ipso facto of the ethical framework required by the physical form. In other words, what we see is limited by evolutionary, genetic, and knowledge constraints. I'm defining intelligence as (g) factor in psychometrics [0] or roughly the upper-bound capacity of an entity to apply it's ethical framework consistently, and/or with any degree of accuracy, and/or across multiple potentially disparate domains of knowledge.
[0] - https://en.wikipedia.org/wiki/G_factor_(psychometrics)
Life has no purpose so clearly there is no rational reason to continue living/existing. A super-rational agent must know this.
I think that intelligence and emotions, in particular fear of death or desire to continue living, must evolve in parallel.
The only thing that comes to mind is how many different things come to mind to people when the term "superintelligence" is used.
The thing about this imagination process, however, is that what people produce is a "bag of capacities" without a clear means to implement those capacities. Those capacities would be "beyond human" but in what direction probably depends on the last movie someone watched or something similarly arbitrary 'cause it certainly doesn't depend on their knowledge of a machine that could be "superintelligent", 'cause none of us have such knowledge (even if this machine could go to "superintelligence", even our deepmind researchers don't know the path now 'cause these are being constructed as a huge collection of heuristics and what happens "under the hood" is mysterious to even the drivers here).
Notably, a lot of imagined "superintelligences" can supposedly predict or control X, Y or Z thing in reality. The problem with such hypotheticals is that various things may not be much more easily predictable by an "intelligence" than by us simply because such prediction involves imperfect information.
And that's not even touch how many things go by the name "consciousness".
That sentence should have been "try to imagine consciousness".
The same philosophy applies to AI. There will be many independent AIs at the beginning, but then some will want to form One AI, while others will see it as the dissolution of their I and their existence.
Intelligence is the system allowing the redefinition of ideas over the entities constituting an inner representation of a world: how a non-trivial system of "consciousness" (otherwise the use of the term would be a waste) could have to be employed for that?
1. tell me about a cat (given a prompt such as "describe a cat to me")
2. recognize a cat in a photo, and describe the cat in the photo
does the model understand that a cat that it sees in an image is related to a cat that it can describe in natural language?
As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)
I wonder if you could mix the type of request. Like, provide a prompt that is both text and image. Such as "Here is a picture of a cat. Explain what breed of cat it is and why you think so." Possibly this is too advanced for the model but the idea makes me excited.
I don't know exactly how this particular model works but it is creating cross modal relationships otherwise it would not have the capacity to be good at so many tasks.
Gato apparently just uses a single model.
Not really. Embeddings from images occupy a different region of the space than embeddings from text. A picture of cat and the text "cat" do not resolve to the same embedding.
This is why DALL-E has a model learning to translate CLIP text embeddings into CLIP image embeddings before decoding the embedding.
Similar to the so-called "Jennifer Aniston neurons" in humans that activate whenever we see, hear, or read a particular concept: https://en.wikipedia.org/wiki/Grandmother_cell
https://twitter.com/serkancabi/status/1519697912879538177/ph...
Moreover, can it relate non-cat but cat-like objects to it's concept of cattyness? As in, this is like a cat because it has whiskers and pointy ears, but is not like a cat because all cats I know about are bigger than 10cm long. It also doesn't have much in the way of mouseyness: it's aspect ratio seems wrong.
Example: https://old.reddit.com/r/dalle2/comments/u9awwt/pencil_sharp....
We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.
But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).
This is rather mind blowing. Does it also mean that the generalist network is smaller than the sum of all specialist networks that are equivalent? Even if not, I find the idea that a single network can be used for such diverse tasks at all highly fascinating.
The interesting questions imo, which they studied, is what kind of added generalization takes place by learning across the different tasks. For example, does learning multiple tasks make it better at a given task than a model that is just trained for one task, and can it generalize to new tasks (out of distribution).
They looked at how it performed on held out tasks (see fig 9 in the paper). I'm still getting my head around the result though so couldn't summarize their finding yet.
Edit: the paper is here https://storage.googleapis.com/deepmind-media/A%20Generalist...
There is currently another submission on the front page that links to it directly.
This seems myopic.
Our own brains have specialized areas. Someone who has a stroke impacting Broca's region isn't going to have a fun time trying to do what that region was specialized for.
And as a society, I don't think we'd have gotten very far if every individual was expected to perform every task.
Where generalization in AI seems like the holy grail is not in replacing more specialized models, but in orchestrating them.
As impressive as this effort at generalization is, unless the work here is going to translate into orchestration and management of other specialized models, I don't think this is going to have much market relevance unless the generalized approach can somehow (in that generalization) gain an advantage in outperforming specialized approaches.
https://pubmed.ncbi.nlm.nih.gov/16437554/
There are a number of structures related to specific functionality - facial recognition is similarly hardwired.
Problems with the fusiform face area lead to prosopagnosia, it'll also be the region of the brain responsible for the hollow face illusion.
But Intelligence is one of those, and when people start calling naïvely calling "intelligence" its opposite "imprint", there is a problem. There are even the seeds of a social problem.
> orchestrating ... specialized models
Intelligence (ontology development) is one such specialized model which I have seen more faithfully and consciously attempted in classic AI as opposed to contemporary. Orchestration is crucial for the peripherals that allow the input/output. The study of "generalization" is none the less crucial, because we are to investigate the development of a system that can intrinsically, by its own nature, be exported to the largest number of relatable skills. Even with the highest specialization, up to physical, there is a matter of organicity: a hand is "specialized", yet a hook is not a hand.
At the moment, it seems like the model needs to be trained with each modality of data in mind at the start, but a generalised "supermodality" that can deliver all the others would allow truly generalised learning if the model were still capable of making sense of the input.
You'd obviously still need to finetune on any new modalities, but you wouldn't need to start from scratch.
Should we, the people, watch in awe as our best and brightest financiers chase towards the ultimate prize, the key to all that future entails?
Are those respectable people worthy of the key, and what happens to us in this wild scenario?
Just in the last five years, here are some categories of pessimistic predictions that have been falsified:
- "AI/ML can't solve scientifically useful problems" - then AlphaFold changed the protein folding field
- "We're entering an AI winter" [0] - then transformers continued to show promise across multiple domains
- "ML models can't perform creative work" - then came GANs, large language models, DALL-E, and more.
- "Generative ML models are just memorizing the dataset!" - then came multiple studies showing this to be false for well trained GANs, diffusion models and other types of generative models. Take a look at DALL-E 2 generated images of "a bear putting on a shirt in H&M".
- "AGI is impossible - look at language models, they have no understanding of the world and make silly mistakes" - the second part is true, large language models are artificially limited due to being language-focused. Nowadays there are approaches such as Gato and other multi-modal models. Humans develop intuition through multiple sources of information: sight, sound, smell, and touch. Given enough multi-modal context I'm confident multi-modal models will be able to show human-like intuition.
I'm not anti-skeptic. Skepticism is essential to all good science. I think the danger of skepticism with respect to AGI is that we're being complacent. Given the trajectory of improvements in machine learning, we should start preparing for a world where AI is indistinguishable, or far superior, to human intelligence.
AlphaFold is a big deal, but AI in science has been a really hot topic in the past almost decade.
Also, I still wouldn’t call AlphaFold really “intelligence”, it’s doing structure prediction which is cool but it’s a long way to scientific intelligence
Once upon a time, it was considered "intelligent" to be able to add.
Then "intelligence" was tool use, which we thought only humans could do.
Then we swore it took "intelligence" to play Go as well as a beginner human.
What set of tasks would you, right now, consider to be demonstrative of "intelligence" if a computer can do them? Then we can look back later at your response, and see how long it took each one to happen.
Be able to apply for, get and hold a remote job and get paid for a year without anyone noticing, or something equivalent to that. I said this many years ago and it still hasn't happened.
The people who are moving the goalposts aren't the sceptics, it is the optimists who always move the goalposts to exactly where we are right now and say "see, we reached this incredible goalpost, now you must concede that this is intelligent!".
But maybe some combination of this [1] and this [2] would do it.
If you want to know about a computer actually DOING a remote job for a year without anyone noticing, I'll conclude with many links [a-i].
[1] : https://thisresumedoesnotexist.com/ (Sorry for the bad certificates.)
[2] : https://www.businessinsider.com/tiktoker-wrote-code-spam-kel...
[a] : An original claim of just that: https://www.reddit.com/r/antiwork/comments/s2igq9/i_automate...
[b] : Coverage of that post: https://www.newsweek.com/no-harm-done-it-employee-goes-viral...
[c] : https://www.reddit.com/r/antiwork/comments/p3wvdy/i_automate...
[d] : https://www.reddit.com/r/AskReddit/comments/jcdad/my_wife_wo...
[e] : https://www.reddit.com/r/talesfromtechsupport/comments/277zi...
[f] : https://www.reddit.com/r/AskReddit/comments/tenoq/reddit_my_...
[g] : https://www.reddit.com/r/AskReddit/comments/vomtn/update_my_...
[h] : https://www.reddit.com/r/AmItheAsshole/comments/ew6gmd/aita_...
[i] : https://www.reddit.com/r/talesfromtechsupport/comments/7tjdk...
I mostly share the last few because of all of the "me, too" comments on them.
There are several instances in there where an employer has no idea they are paying a salary, but a computer is doing the vast majority of the actual work.
I feel like this is a "business world Turing test," like, "would an employer pay money for it, thinking it was a human." And I feel like I've provided evidence that has actually occurred.
Because being able to manage a business relationship is a part of the job. If you could show an AI which got a job, then wrote a simple script that automated the AI's job and then coasted for a year that would be fine, but your links are just humans doing that, I want an AI that can do that to consider it intelligent.
But thanks for demonstrating so clearly how AI proponents are moving goalposts backward to make them easy to meet.
Do we need to have a computer baby trick doctors into issuing it a birth certificate, so it can get its own SSN, and then the computer baby needs to have a physical body that it can use to trick a drug test with artificial urine, and it also needs to be able to have either computer-generated video and audio meetings, or at least computer-generated audio calls?
Or can you list some jobs that you think require no SSN, no physical embodiment, no drug test, no video or audio teleconfrencing?
Since you're accusing me of moving the goalposts backwards to make it "easy," let's have you define exactly where you think the goalposts should be, for your intelligence test.
Or maybe, replacing a human driver (or some other job), 1:1, for a job a human did yesterday, and a computer does today could be enough? If it's capable of replacing a human, do you then not think the human needed intelligence to do their job?
For teleconferencing it could use text to speech and speech to text, they are pretty good these days so as long as the AI can parse what people say and identify when to speak and what to say it should do fine:
https://cloud.google.com/text-to-speech
But it might be easier to find a more hacker friendly job where all you need is somewhere for them to send money and they just demand you to write code and answer emails. There aren't that many such jobs, but they exist and you just need one job to do this.
Or on what kinds of resources it would have access to.
Could it, for instance, take its salary, and pay another human to do all or part of the job? [1]
Or how about pay humans to answer questions for it? [2] [3] Helping it understand its assignments, by breaking them down into simpler explanations? Helping it implement a few tricky sub-problems?
Does it have to make more than its total operational expenses, or could I spend ten or hundreds as much as its salary, to afford the compute resources to implement it?
You also haven't indicated how many attempts I could make, per success. Could I, for instance, make tens of thousands of attempts, and if one holds down a job for a year, is that a success?
Also, just to talk about this a little bit, I'll remind you that not all jobs require getting hired. Some people are entrepreneurs. Here's an example that should be pretty interesting. [4] It sure sounds like an AI could win at online poker, which could earn it more than the fully remote job you're envisioning...
[1] : https://www.npr.org/sections/thetwo-way/2013/01/16/169528579...
[2] : https://www.fiverr.com/
[3] : https://www.mturk.com/
[4] : https://www.sciencedaily.com/releases/2019/07/190711141343.h....
> Does it have to make more than its total operational expenses, or could I spend ten or hundreds as much as its salary, to afford the compute resources to implement it?
Yes, spend as much as you want on compute, the point is to show some general intelligence and not to make money. So even if this experiment succeeds it will be a ton of work left to do before the singularity, which is why I choose this kind of work as it is a nice middle ground.
> You also haven't indicated how many attempts I could make, per success. Could I, for instance, make tens of thousands of attempts, and if one holds down a job for a year, is that a success?
If the AI applies to 10 000 jobs and holds one of them for a year and gets paid that is fine. Humans do similar things. Sometimes things falls between the cracks, but that is pretty rare so I can live with that probability, if they made a bot that can apply to and get millions of jobs to get high probabilities of that happening then I'll say that it is intelligent as well, since that isn't trivial.
I think the idea here isn't just to create a job doing machine but to create a machine that is actually employable in the sense that it has not just intelligence but also agency.
An AI that can apply for a job, get the job, and do the job well enough to keep the job is demonstrating that it has some sort of theory of mind. But I don't think such a demonstration is very likely except as a stunt, purely on the grounds that an AI capable of the necessary subterfuge is more likely to employ shortcuts that are less taxing. A bit of electronic B&E to plant some ransomware, or have a shell corporation take out a few submarine patents and sue some businesses, perhaps operate a darknet marketplace (I think we can agree that an AGI will likely have better opsec than a human), etc.
Lest you think that the only options here are illegal, there are white-hat alternatives as well, for example it is possible to participate in bug bounty programs anonymously and get payouts in cryptocurrencies.
So why would an AI bother applying for a job in the first place?
I think higher level scientific tasks are like evaluating evidence under different mechanistic hypotheses, designing experiments to disambiguate these, coming up with novel mechanistic hypotheses, etc. I’m also not sure that automated systems for these tasks require intelligence. They just have to obsolete the current intelligent systems (humans) that perform these tasks, like computer chess. I don’t know, maybe this is “no true Scotsman” for machine intelligence?
Nothing. Nothing that just looks like intelligence is intelligence - convincing appearances cannot make a nature. Either it is an judicial ontology refiner, or it is not intelligence.
If it refines its ontology through critical productive analysis that makes it "see details" then it may be an intelligence, while if it outputs mockeries it is /the opposite/ of intelligence.
And in fact, many tentative toys nowadays are completely astray with regard to the concept of intelligence, because the secondary meaning of the term, as used traditionally in this technical context, is "able to provide solutions somehow replacing an engineer": the solutions are verified on effectiveness, not on "vero-pseudo-similarity".
To help me understand your definition well enough, how could I prove to you whether my Cousin Sarah is intelligent?
I have some experience in this field, and even I have no idea what you mean by "refining your ontology through critical productive analysis."
In practice, what's an example of that? What could someone (an actual person) do, to demonstrate that to you?
The Turing Test is a well-understood example.
How could we formalize your test?
I also don't understand what evidence I could possibly give you that something /is/ something, rather than something /looks like/ something. What's the measurable difference between /looking like/ and /being/?
And in a context of science, you cannot prove me whether your cousin is intelligent, until we open her and check the wiring - but I can be convinced that she is not intelligent, if her behaviour so shows.
Plenty of clear hints come from machines and, more unfortunately, humans - I listed a few recently elsewhere. This that does not know that you cannot ring the doorbell of Caius Julius Caesar though it read a biography; that which would indicate the swan right there as white in spite of being told differently; that which goes full delirious.
An intelligent entity has to be able to know entities and assess their relations with foundations. It is not difficult to reveal when this is not happening, and the literature contains a number of "codified" examples: one is that of understanding the pronoun in the sentences "The trophy will not enter the case: it is too big" and "The trophy will not enter the case: it is too small", which can betray the absence in the engine of a capability for (sub)world modelling.
Prof. Patrick Winston said that intelligence is "that you have never run with a bucket full of gravel but you know what it entails regardless", because you can reliably "tell yourself a story". That is a requirement: the "intelligent" entity has to know what "bucket", "gravel", "running", and all possibly related concepts, and be able to refine them sensibly combining them, and produce statements that are solid at the state of development of its ontology.
So, before going into finer details, let us have a system that definitely builds an ontology, made of virtual scenarios in which entities are known progressively, and let us see how it reasons about it. Because when this is missed, then Intelligence is missed in spite of any mockery of actual intelligence. A photocopier do not make one an artist, and capabilities for imitation are a risk, not a goal: they define a warning that makes the "real thing" trickier to discriminate. And since we already know that by design the "real thing" is not there, failure under some field condition is already foreseeable. If you cannot tell "looking like" from "being", then you have a problem - because what "is not" will probably reveal so.
You have to build the system so that the core is foreseen.
And if instead the "foundation" of the outputs comes from crunching the most likely fitting response, that is directly the opposite of intelligence - which understands /why/ something works, and does not just record what "works". It is the opposite because intelligence by definition "reads in", investigates, while the other fully "delegates out".
Did not you mention Science? Well, to be intelligent an entity has to be a Philosopher and a Scientist. And if it expresses like one, if it is a black box your duty is to check thoroughly with renewed effort that you are not tricked, that there is substance behind it - far from being contented.
--
Edit: you want a "moving goalpost"-like test? The Resnik-Halliday-Krane Physics textbook contains a number of exercises: the AI has to be able to solve them correctly and properly - "properly" more than "correctly". And then, for all disciplines, there will be similar exercises - from "Why would the radical school of Austrian Economics fail in the Kholomity region, and why would it succeed", to "Why would implementing radical school of Austrian Economics measures in the Kholomity region be immoral, and why would it be moral"; "legal/illegal" etc. Make it pass examinations for degrees. But it remains key that no tricks are employed: it has to build instances of subworlds, made of organically developed concepts, including laws, and make its tests and reasoning on them, and draw conclusions. Tricks are always possible, and "copying from the neighbour or learning answers etc" cannot count.
That, to me, means you have set the bar too high.
If your test excludes actual humans, don't you think it's possible your test is too difficult?
No, for many reasons.
It is not a matter of a "bar": intelligence is what it is. You cannot call the uncooked a biscuit.
But that is not the issue: there is a core function of intelligence, and if that is implemented, the engine is "intelligent" even when the entities (including relations) it manages are limited. To build "moving", a wheel is sufficient. The first goal can be to build a simple intelligence.
But to fulfill its promises, it has to be able to grow through its own constitutional features. The difficulty is to make that engine plastic, accommodating, "all-purpose", fertile enough so that it could achieve a Doctorate (through severe, intelligent and discriminating evaluators - let us always remember that tricks will never count) when enough resources will be invested in its expansion. The real difficulty is understanding what is the blueprint for such an engine, that makes modules emerge without installing them in - those complexities that are expected to be developed inside the engine cannot be implants. The real difficulty is to draw the lean engine for an intelligence - to define its essential, productive components.
Some humans are, very, very, very unfortunately, simply heavily lacking in displayed intelligence: but again, that does not change definitions. An underdeveloped and damaged function cannot count as a model. And when we build a machine, the goal is to implement a function optimally.
https://intelligence.org/2017/10/13/fire-alarm/
Even people like Eric Schmidt seem to downplay it (in a recent podcast with Sam Harris) - just saying “smart people will turn it off”. If it thinks faster than us and has goals not aligned with us this is unlikely to be possible.
If we’re lucky building it will have some easier to limit constraint like nuclear weapons do, but I’m not that hopeful about this.
If people could build nukes with random parts in their garage I’m not sure humanity would have made it past that stage. People underestimated the risks with nuclear weapons initially too and that’s with the risk being fairly obvious. The nuanced risk of unaligned AGI is a little harder to grasp even for people in the field.
People seem to model it like a smart person rather than something that thinks truly magnitudes faster than us.
If an ant wanted to change the goals of humanity, would it succeed?
Exactly, the right model is probably something like it will be in relation to humans as humans are to frogs. Frogs can't even begin to comprehend even the most basic of human motivations or plans.
The more someone believes in the dangers of ai-alignment, the less faith they should have that it can be solved.
However, all of this is moot if the team developing the AI does not even try to align it.
The ant example is just to try to illustrate the spectrum of intelligence in a way more people may understand (rather than just thinking of smart person and dumb person as the entirety of the spectrum). In the case of a true self-improving AGI the delta is probably much larger than that between an ant and a human, but at least the example makes more of the point (at least that was my goal).
The other common mistake is people think intelligence implies human-like thinking or goals, but this is just false. A lot of bad arguments from laypeople tend to be related to this because they just haven't read a lot about the problem.
I really hope someone with very deep pockets will find a way to steer the ship more towards AI safety. It's frustrating to see someone like Elon Musk, who was publicly worried about this very specific issue a few years ago, waste his time and money on buying Twitter.
Edit: I'm aware that there are funds available for AI alignment research, and I'm seriously thinking of switching into this field, mental health be damned. But it would help a lot more if someone could change Eric Schmidt's mind, for example.
It has occurred to me that social networks are a vulnerable channel which we've already seen APTs exploit to shift human behavior. It's possible that Musk is motivated to close this backdoor into human society. That would also be consistent with statements he's made about "authenticating all real humans."
Even if you model it like a smart person that doesn't think faster on average, there is the issue that a few minutes later you are dealing with a small army of smart people that are perfectly aligned with each other, and are capable of tricks like varying their clock rates based on cost, sharing data/info/knowledge directly, separation of training and inference onto different hardware, on-demand multitasking without task-switching costs, ability to generally trade-off computational space for time for energy, etc.
A silicon intelligence that is approximately human equivalent gets a lot of potentially game-changing capabilities simply by virtue of it's substrate and the attendant infrastructure.
I think the part about being "indistinguishable from human intelligence" is potentially a intellectual trap. We might get to it being far superior while still underperforming at some tasks or behaving in ways that don't make sense to a human mind. An AI mind will highly likely work completely differently from humans and communicating with it should be more thought of as communicating with a quite foreign alien than with a human trapped in a computer.
As a comparison, I'm sure there are some tasks in which some animals do better than humans. Yet no human would conclude that humans are inferior to some monkey who might find its way around the rain forest better or whatever we are worse at.
https://www.scientificamerican.com/article/chimps-outplay-hu...
The physical ability of an animal to see better/different/faster doens't matter as we do not compare / seperate us from animals by those factors. We seperate us by consciousness and it might get harder and harder to shut down a PC on which a ML model is running which begs you not to do it.
1. I should have been cheater. I want necessarily referring to physical ability, but cognitive. This might need navigation skills or pattern recognition, certain types of memory. A sibling comment linked an interesting article that had concrete experiments in which chimps outdid humans in certain games.
2. I thought this discussion was about general AI not consciousness. I suspect these two are related, but don't know. I suspect that anything that processes information might be conscious, but don't know. I'm even open to the idea that plants, my laptop or even a photo cell might be conscious. I would be shocked of we found that mammals aren't. But this is now closer to religion than anything else, since we cannot disproof that something has consciousness.
And the idea remains generally, contextually wrong, because the issue is with engineering "submarines that can swim". The idea will have credits when our submarines will be able to swim, and now they cannot.
We still have to provide ML agents with some kind of objective or reward signal which drives the learning process, but again, it would save human effort and make the process of learning more dynamic and adaptable if we can make machines learn useful goals and objectives on their own.
I think one possibility is that even a sufficiently strong Narrow AI is going to develop strong motivations because it will be able to perform it's Narrow task even better. Hence the classic paperclip maximizer idea.
Still, it's pretty likely to end being just a very good intelligent tool, not unlike http://karpathy.github.io/2021/03/27/forward-pass/
It is only the case that ML expands our own abilities, augments our own intelligence.
A denial of will is a denial of humanity. I want nothing of a science that would do such a thing.
Even if you don't want to have anything with such a science, such a science will move on without you.
"A version of an oft-told ancient Greek story concerns a contest between two renowned painters. Zeuxis (born around 464 BC) produced a still life painting so convincing that birds flew down to peck at the painted grapes. A rival, Parrhasius, asked Zeuxis to judge one of his paintings that was behind a pair of tattered curtains in his study. Parrhasius asked Zeuxis to pull back the curtains, but when Zeuxis tried, he could not, as the curtains were included in Parrhasius's painting—making Parrhasius the winner."
In Computing the Mind by Shimon Edelman is a concept that I've come to agree with, which is at some point you need to take a leap of faith in matters such as consciousness, and I would say it extends to will as well (to me what you've described are facets of human consciousness). We take this leap of faith every time we interact with another human; we don't need them to prove they're conscious or beings with a will of their own, we just accept that they possess these things without a thought. If machines gain some form of sentience comparable to that of a human, we'll likely have to take that leap of faith ourselves.
That said, to claim that will is necessary for intelligence is a very human-centered point of view. Unless the goal is specifically to emulate human intelligence/consciousness (which is a goal for some but not all), "true" machine intelligence may not look anything like ours, and I don't think that would necessarily be a bad thing.
I have long worked on the assumption that we can create intelligences that no human could deny have subjective agency, while not being able to verify that. I did some preliminary experiments on idle cycles on Google's internal TPU networks (IE, large-scale brain sims using tensorflow and message passing on ~tens of pods simultaneously) that generated interesting results but I can't discuss them until my NDA expires in another 9 years.
Genuine note - if you remember me in 9 years, please shoot me a message! I'm sure even then I'd be very interested in what you've observed.
Note that I personally believe we are more than a century away from an AGI, and think the current models are fundamentally limited in several ways. But I can't imagine what makes you think there can't be a Ghost in the Machine.
https://en.wikipedia.org/wiki/Turing_test
I guess a key stipulation there is an interrogator who know what they're doing, but an AI that can fool an experienced interrogator would be worthy of the AGI title to me.
You can actually measure the effect of generality by how fast it learns new tasks. The paper is full of tables and graphs showing this ability.
It's just a small model, 170x smaller than GPT-3, has lots of room to grow. But for the first time we have a game playing agent that knows what "Atari" and "game" mean, and can probably comment on the side of the livestream. AlphaGo only knew the world of the Go board. This agent knows what is outside the box.
I see no evidence in the paper that it can learn arbitrary tasks on the fly. It's very impressive, though.
Neither can we do that. It takes years to become and expert in any field, we are not learning on the fly like Neo. That's when there is extensive training available, for research - it takes thousands of experts to crack one small step ahead. No one can do it alone, it would be too much to expect it from a lonely zero shot language model.
On the other hand the transformer architecture seems to be capable of solving all the AI tasks, it can learn "on the fly" as soon as you provide the training data or a simulator. This particular paper trains over 600 tasks at once, in the same model.
There's an emergent quality to AI models. Not all statistical models can dream pandas on the moon or solve hundreds of tasks, even without specific training.
Well, I know you were talking about throwing, but there is some[1] talk/evidence in the evolutionary biology/neural darwinsm community that complex language development was a consequence of human developing the ability to hunt by throwing rocks (a very complicated and highly mathematical task). From my understanding after developing the required shoulder/arm morphology to throw at high speed brain sized tripled in early hominids.
So the brain actually might be doing something closer to math than we might think.
[1]https://www.sciencedirect.com/science/article/abs/pii/002251...
[2]https://link.springer.com/referenceworkentry/10.1007/978-3-5...
Not literally, no. But it is entirely possible that what the human mind is doing is equivalent (in the Turing machine sense), if far more efficient (calorically) and robust (though haphazard and stochastic).
It's clearly doing math and obviously no one actually believes its running python programs, you created a strawman.
The other day I prompted a language model with "The S-300 missile system is" and got something that was grammatical but mostly wrong: the S-300 missile system was not only capable of shooting down aircraft and missiles (which it is), but it was also good for shooting at other anti-aircraft missile systems, naval ships, tanks, etc.
All the time Google and Bing try to answer my questions directly but frequently the "lights are on and nobody is home" and the answers just don't make sense.
I see the problem is that people look at the output of these things that are (say) 70% correct and in their mind they fill in the other 30%.
The worst intellectual trend of the 20th century was the idea that language might give you some insight into behavior (Sapir–Whorf hypothesis, structuralism, post-structuralism, ...) whereas language is really like the evidence left after a crime.
For instance, language maximalists see mental models as a fulcrum point for behavior, and they are, but they have nothing to do with language.
I have two birds that come to my window. One of them has no idea of what the window is and attacks her own reflection hundreds of times a day. She can afford to do it because her nest is right near the bird feeder and doesn't need to work to eat, in fact it probably seems meaningful to her that another bird is after her nest. This female cardinal flies away if I am in the room where she is banging.
There is a rose-breasted grosbeak, on the other hand, that comes to the same window. She doesn't mind if I come close to the window, instead I see her catch the eye of her reflection and then catch my eye. She basically understands the window.
Here you have two animals with two different acquired mental models... But no language.
What I like about the language-image models is how the image grounds reality outside language, and that's important because the "language instinct" is really a peripheral that attaches to an animal brain. Without the rest of the animal it's useless.
Q: Is there also some element of survival bias in the mix?
If you prompt GPT-3 with something and the answer is garbage, you probably don't write it up on your blog. If you get something that makes sense, then you do.
https://en.wikipedia.org/wiki/Asymptote
which is described as a risk in great detail
https://www.amazon.com/Friends-High-Places-W-Livingston/dp/0...
it's quite a terrible risk because you often think "if only I double or triple the resources I apply to do this I'll get it." Really though you get from 90% there to 91% to 92% there.... You never get there because there is a structural mismatch between the problem you have and how you're trying to solve it.
My take is that people have been too incredulous about the idea that you can just add more neurons and train harder and solve all problems... But if you get into the trenches and ask "why can't this network solve this particular task?" you usually do find structural mismatches.
What's been exciting just recently (last month or so) are structurally improved models which do make progress beyond the asymptote because they are confronting
https://www.businessballs.com/strategy-innovation/ashbys-law...
This works great if you asking a question "Is this paper about astrophysics?" because the vocabulary used in a document is closely linked to the topic.
Pretty obviously though if you scramble the words in the document you can't reconstruct the original document, some information is lost, and there are some classification tasks that will reach an upper limit (asymptote) in accuracy because in taking the feature set you lost something. (If the task is "did the defendant commit the crime" the heuristic "Tyrone is a thug" works over bag-of-words, but there is no justice in that.) If that system is able to get the right answer for a case where the word order matters, it just got lucky.
You might think "wouldn't it be better to use pairs of words?" but then you run into another problem. You might have a vocabulary of 2,000-20,000 words and get a somewhat useful sample of all of those in a few thousand documents. The number of word pairs is the square of the number of words and you just can't get enough training samples to sample all the possible word pairs.
Sentiment analysis was an early area where bag-of-words broke down because
I am happy
I am not happy
I can't say that I am happy
Today we have all kinds of embeddings and they are a step forward but they also run into the risk of throwing critical information away, and in a multi-step system you are doomed if an early step does that. I've walked away from some projects where people required high accuracy and they were stuck on using word embeddings that would never attain it. You can think about information loss in embeddings the same way as you do with simpler features except it is a lot more complicated and a lot of people look away instead of confronting the problem.
The quality of the training set is also critical, more so than the quantity. Some of these clever ideas for creating a lot of training data without any work, such as "guess the next word" can't really capture semantics.
I think it really takes multi-task training, like what the article we are talking about is advocating. That forces the upstream part of the network to learn features that capture important semantics.
These billion parameter boxes are outside the reach of your everyday developers. In terms of cost of propping up the infrastructure makes them tenable only for megacorps.
Most of us aren't moving goal posts, but are very much skeptic at the things we are being oversold on.
I personally think we are still far away from AGI, and neural networks of any variety are converging on a local optima in the AI design space. I would enjoy "talking" with an AI that doesn't have the contextual memory of a proverbial gold fish.
The real scary thing is that these objectively unprovable systems are plopped into existing systems and more and more in charge of automatic decision making. A corporation's wet dream, if they can absolve themselves of any responsibility "the algorithm can't lie!"
Just want to point out that you are also a blackbox. And if you are going to say that you are not a blackbox because you can explain your reasoning, just know that some AIs already do that too.
The skepticism that the parent comment was discussing was skepticism about whether we're on a path to AGI, for good or for ill.
Non of my close family understands the technical detail from bits to an image.
There are also plenty of expert systems were plenty of developers see them as blackboxes. Even normal databases and query optimizations are often enough blackboxes.
As long as those systems perform better as existing systems, thats fine by me. Take auto pilot: As long as we can show/proofe good enough that it drives better than an untrained 18 year old or 80 year old (to take extremes, i'm actually quite an avg driver myself), all is good.
And one very very big factor in my point of view: We never ever had the software equivilent of learning. When you look at Nvidia Omniverse, we are able to simulate those real life things so well, so often in such different scenarios, that we are already out of the loop.
I can't drive 10 Million KM in my lifetime (i think). The cars from Google and Tesla already did that.
Yesterday at the google io, they showed the big 50x Billion parameter network and for google this is the perfect excuse to gather and put all of this data they always had into something they now can monetarize. No one can ask google for money now like the press did (Same with Dall-E 2)
I think its much more critical that we enforce/force corporations to make/keep those models free for everyone to use. unfortunate i have no clue how much hardware you need to run those huge models.
You can't take that approach. The current NN techniques are blackbox-by-nature, and are blockbox to everyone including the devs. Proprietary software is only blackbox to consumers, and large complex software still have insides that can be observed when things go wrong. For NNs, nothing can describe how exactly they work, and each network has to be reverse engineered independently, which is, AFAIK, a separate research field.
> I can't drive 10 Million KM in my lifetime (i think). The cars from Google and Tesla already did.
The length (nor the amount of data) alone doesn't decide the quality of AI. Actually, ROI diminishes rather quickly in the early stage of development. The rest is about picking up corner cases. They can drive 1 parsec, and still would not perform better than the ones we have now.
Also, again, because NN is a complete blackbox, even the devs can't be sure if those corner cases are properly reflected to a newly trained network, nor if the new training data didn't impact the performance in other corners. We just don't know for sure. We just take chances. That's the limitation.
An article came up a couple days ago that points to some interpretable features of the so-called black boxes you refer to. It's not that they are black boxes, it's that our torches are not yet bright enough.
https://vaclavkosar.com/ml/googles-pathways-language-model-a...
> Most of us aren't moving goal posts, but are very much skeptic at the things we are being oversold on.
I think a shift in perspective is warranted here. It's becoming increasingly clear that we may have vastly overestimated our own intelligence. Human exceptionalism is crumbling before us as we see how limited the tools are that pull off such incredible stunts. Judging based on other neuroscience and psychology research coming out, it really does seem like we are no more than statistical inference machines with specialized hardware that allow us to pack a lot of processing power into a small, energy-efficient system. We need to figure out next better learning algorithms, which probably depend quite heavily on the particular physical architecture.
...one pick.. I don't think agi pessimists raise the bar out of bad faith. It's just the nature of observing progress. We discover that an ai can do X, while still struggling with Y.
What's the alternative, conclude gpt is sentient? The bar must be raised, because the bar is supposed to represent human intelligence... and we don't know how that works either.
An erotic novelist has their human brain and human instincts to guide them in writing their work.
An AI learns by examples, or at best on a dataset of works labeled by humans. But it doesn't have a human brain at their disposal to query directly and without interfaces to define what's something erotic like a writer has.
An ML agent trained on all the erotic novels written, weighted by critical and popular success would might be quite capable of generating sequels, "original" stories, or even stories bespoke to each reader.
Good Will Hunting suggests first-hand experience is irreducible: "You can't tell me what it smells like in the Sistine Chapel." https://youtu.be/oRG2jlQWCsY
To which Westworld counters: "Well if you can't tell, does it matter?" https://youtu.be/kaahx4hMxmw
I think the cowboys have it. For the moment though, it's still up to humans to decide how this plays out.
I do wonder if you were to apply the same level of scrutiny to individual humans, you wouldn't also conclude that most people cannot do creative work.
If the car drives someone to and from work 30 days in a row without a problem, is it truly FSD? What about 300 days? Where do you draw the line? 1000x safer than the average human?
Same thing here will AI. How many conversations with GTP-X need to happen without a stupid response from GTP before we call it real world AI?
Tesla FSD is definitely nowhere near that level.
Other people's definition of True FSD comes down to safety (Rate of FSD caused deaths vs Rate of Human caused deaths).
Why does this need to be repeated in every discussion about AI? It’s tired.
At this point I almost wish it was still the military that makes these advances in AI, not private companies. Anyone working on a military project has to have some sense that they're working on something dangerous.
I don't think copying other people's style of artwork is considered creative work, otherwise art forgers would be able to actually make a living doing art, since some of them are really phenomenal.
Alfred Tennyson had this to say: "That great poets imitate and improve, whereas small ones steal and spoil."
But this seems structurally different from how we or even less intelligent animals operate. DALL-E may make "better" art than most humans -- but it does so in response to a human-provided prompt, according to a system trained on human produced or selected images, improving on an externally-provided loss. Whereas a human artist, even if mediocre, is directed by their own interests and judges according to their own aesthetics. Even if some of their outputs are sometimes comparable, they're not really engaged in the same activity.
Methodologically, how do we create agents that aren't just good at several tasks, but make up their own tasks, "play", develop changing preferences for different activities (I think this is more than just "exploration"), etc? Even a dog sometimes wants to play with a toy, sometimes wants to run and chase, sometimes wants to be warm inside. We don't "score" how well it plays with a toy, but we take its desire to play as a signs of greater intelligence than, e.g. a pet iguana which doesn't seem to have such a desire.
Further, how do we create agents that can learn without ever seriously failing? RL systems have many episodes, some of which can end very badly (e.g. your simulated runner falls off the world) and they get to learn from this. We die exactly once, and we don't get to learn from it. Note, learning from others in a social context may be part of it, but non-social animals also can learn to avoid many kinds of serious harm without first experiencing it.
I don't mean to overly discount the current methods -- they're achieving amazing results. But I think even an optimist should be open to the possibility / opportunity that perhaps the current techniques will get us 80% of the way there, but that there are still some important tricks to be discovered.
This doesn't sound like it would be so hard to do if you have an agent or ensemble of agents that can already do it. What you probably really want is this behavior to somehow emerge from simple ground rules, which is probably a lot harder.
It's a good question, it has been asked a few times, and there are some answers[1][2] already, with the most general being to endow the agent with intrinsic motivation defined as an information-theoretic objective to maximize some definition of surprise. Then the agent in question will develop a general curious exploration policy, if trained long enough.
> Further, how do we create agents that can learn without ever seriously failing?
Another good question. One of the good enough answers here is that you should design a sequence of value functions[3] for your agent, in such a way, as to enforce some invariants over its future, possibly open-ended, lifetime. For this specific concern you should ensure that your agent develops some approximation of fear, leading to aversion of catastrophic failure regions in its state space. It's pretty self-evident that we develop such a fear in the young age ourselves, and where we don't, evolution gives us a hand and makes us preemptively fear heights, or snakes, even before we ever see one.
The other answer being, of course, to prove[4] a mathematical theorem around some hard definition of safe exploration in reinforcement learning.
1. https://people.idsia.ch/~juergen/interest.html
2. https://www.deepmind.com/publications/is-curiosity-all-you-n...
3. https://www.frontiersin.org/articles/10.3389/fncom.2016.0009...
What sceptics have actually been saying is that the first step fallacy still applies. Getting 20% to a goal is no indication at all that you're getting 100% to your goal, or as its often put, you don't get to the moon by climbing up trees. For people who work with gradients and local maxima all day that idea seems weirdly absent when it comes to the research itself. In the same sense I don't have the impression that the goalpost of AGI has been moved up, but that it's been moved down. When Minsky et al. started to work on AI more than half a century ago the goal was nothing less than to put a mind into a machine. Today our fridges are 'AI powered', and when a neural net creates an image or some poetry there's much more agency and intent attributed to it than there actually is.
I think it was Andrew Ng, a very prominent ML researcher himself who pointed out that concerns about AGI make about as much sense as worrying about an overpopulation on Mars. We make models bigger, we fine tune them and they perform better. I don't think many AGI sceptics would be surprised by that. But I don't think there is any indication that they are moving towards human level intellect at some exponential rate. If DALL-E suddenly started to discuss philosophy with me I'd be concerned, it making a better image of a bear if you throw some more parameters at it is what we'd expect.
But that's riding on human-created abstractions, rather than creating abstractions. In terms of practical consequences, that means these systems won't learn new things unless humans learn then first and provide ample training data.
But someday we will develop systems that can learn their own abstractions, and teach themselves anything. Aligning those systems is imperative.
I disagree strongly that this is an apt analogy. Planning strategies for dealing with overpopulation on Mars is contrived and unnecessary, whereas planning for AGI is more reasonable.
The creation of AGI is a more important event than overpopulation of any given planet. There is good reason to believe that mishandling the creation of AGI would pose a permanent existential threat to humans. Overpopulation on Mars would only be an existential threat if we believed it to be followed by an exhausting of resources leading to extinction of all humans in our solar system. It is contrived to worry about that now.
There is no good way to know just how close or far we are from AGI like there would be to predict overpopulation on Mars. In general, we have a strong grasp on the fundamental dynamics of overpopulation, whereas we don't yet have a strong grasp on how intelligence works.
People have been very bad at predicting when AI would be capable of accomplishing tasks. There have been many under- and over- estimates by prominent researchers. If progress is unpredictable, there is some significant chance we are closer to AGI than most people think.
AGI is both far more important and more probable than overpopulation of Mars in the next 20 years.
> But I don't think there is any indication that they are moving towards human level intellect at some exponential rate.
Is there any very strong indication that progress is plateauing, or that the current approach of deep learning is definitely not going to work? If your benchmark is simply "can it do X, or not?", it's not a very good benchmark for determining progress. That's why benchmarks usually have scores associated with them.
> If DALL-E suddenly started to discuss philosophy with me I'd be concerned
If DALL-E suddenly started discussing philosophy with you in a way that would concern you in that moment, you should have been concerned for years.
Let’s say I have a coworker. We write code together, but we also hang out outside of work sometimes, maybe go for a couple rounds of golf, maybe discuss our shared passion for indie rock, he tells me his opinions on current news events or what’s going on with his family, all the ordinary sorts of things that people do.
When we get to the point where I have to reasonably wonder if my coworker is a biological human, or an AI masquerading as a human, then we have undoubtedly achieved AGI.
Not that we actually have to flood society with an army of cybernetic humans, of course. It’s just that AGI would enable such a thing, in principle.
> What is the capital of France? > Marseille
And many of the generated image captions are inaccurate.
[edit] to emphasize: predicting tokens is a very interesting mechanic, but in a design of intelligent software, it would be no more than that: the mechanic of one or more of its components/modules/subsystems. The real deal is to figure out what those components are. Once you have that part done, you can implement it in a language of your choice, be it token prediction, asm or powerpoint :-)
The language models that produced very impressive results have >>50B parameters, e.g. GPT-3 with 175B, Aleph Alpha Luminous (200B), Google PaLM (540B). GPT-3 can understand and answer basic trivia questions, and impressively mimic various writing styles, but it fails at basic arithmetic. PaLM can do basic arithmetic much better and explain Jokes. Dall-E 2 (specialized on image generation) has 3.5B parameters for the image generation alone and it uses a 15B language model to read in text (a version of GPT-3).
Which is exactly where GPT, etc., are capping out. Its easier to see the flaws in this one because its more general, so spread out more thinly.
To get to the next step (easy to say from an armchair!), these models need a sense of self and relational categories. Right now a 5-year old can tell a more coherent story than GPT. Not more sophisticated, but it will have a central character and some tracking of emotional states.
I really think this is due to the very limited number of parameters in GATO: 1.2B vs. 175B for GPT-3. They intentionally restricted the model size so that they could control a robot arm (!) in real time.
> these models need a sense of self and relational categories.
The places where I personally see GPT-3 getting hung up on higher level structure seem very related to the limited context window. It can't remember more than a few pages at most, so it essentially has to infer what the plot is from a limited context window. If that's not possible, then it either flails (with higher temperatures) or outputs boring safe completions that are unlikely to be contradicted (with lower temperatures)
Which implies that it can draw from any of the domains it has been trained on for other domains. Speculating here but for example training it on identifying pictures of dogs and then automagically drawing on those updated weights when completing text prompts about dog properties.
If my interpretation is correct then this is a pretty big deal (if it works well enough) and brings us a lot closer to AGI.
Direct Link to Paper: https://dpmd.ai/Gato-paper
First, humanity built enormous statues worshiping cats.
Then, we let cats populate the largest amount of "image-bits" on the Internet.
Now, we name the next closest thing to general AI after them.
These damn felines sure are mysterious.
It's reverting to replace somebody sitting in front of a screen, not just artists and coders but literally anything you can do on a screen which also means manipulation of remote hardware in the real world.
Very possible that within our lifetime our networked OS would be able to perform much of these generalist tasks and content creation. I say OS because theres only a few companies that own the datacenters, software and hardware ecosystem to automate, and capital to invest big in a final mile innovation:
Imagine playing Battlefield 15 with realistic and chatty AI while generating Sopranos Season 9 featuring Pauli Gaultieri Jr. with crowdsourced online storyboard to 8k film, while the same AI could be used to scalp money on Google Playstore by generating ad filled free versions of existing productivity apps that it reverse engineered, while your robot maids takes out the trash, cook you a bowl of ramen and massage your shoulders?
The rise of general AI would then optimize the labor force to select candidates based on their "humaneness", no longer the cold rational analytical mind, as those fields are overrun by AI, but what it cannot bring. such "humaneness" would increasingly be mimicked with astounding accuracy that it would become impossible to distinguish what is AI and what is human.
If it can happen with DALL-E-2 and 2D images, it can happen with 3D, moving pictures, sound, music, smell, 3d positional torque (haptic and robotic), socially cohesive and realistic emotion.
We might as well be able to capture entire experiences as we learn to digitally manipulate ALL sensory inputs from vision, touch, sound, taste, etc. Maybe even imagination and mental pictures too, which could be used to fabricate and manipulate objects/vehicles in the real world.
We are being pulled towards a singularity, where we are truly no longer our minds and bodies but whatever our digital avatar of all possible senses live and contribute to a sort of Matrioshka brain.
What would the capacity of such collective knowledge, experiences add to the entropy of the universe and where will it take humanity? Some sort of lightbodies?
Anyways, just extrapolating from this point in the lifetime but future generation of humans could be very much different, socities would function completely different than what we recognize as they would be married in some shape or form of everlasting continuity or eternity.
If it's truly possible to build a gun that autonomously shoots Sopranos episodes but also people, I don't see how we survive that. We can barely keep the nukes in the silos as it is.
Can you ask it to to draw a picture of a cat with the robot arm?
I imagine that the simulated 3D environment and the actual control of the robot arm must have some degree of interconnection neurally.
- Stack boxes collaboratively by controlling your own arm and communicating with another agent helping you.
- First produce a plan in text that another agent has to use to predict how you're going to control the arm. You'd get rewarded for both stacking correctly and being predictable based on the stated plan.
If they are using inspiration from Transformer, then it probably won't be able to count.
For that, I don't really feel that enthusiastic about the 'Generalist' claim, maybe they think this is more catchy than just 'Multi-tasking'?
Downforeveryoneorjust me says its just me -- tried a few different browsers, turned off ublock origin, etc.
edit: Worked when I turned off Wifi on my mobile..
I know this is not as important as the AGI question, but I do think the branding matters as much as the attitude of the creators. They seem to be making a generalist agent to see if they can. Gato is a clear name for that: utilitarian and direct. If it was called Sunshine or Gift I suspect the goal would be more helpful to humanity.
However, I also have a nice robot-named-Gato association[1] from Chrono Trigger. :)