Concrete AI tasks for forecasting (2014)
aiimpacts.org
aiimpacts.org
I see some criticism of the poker tasks, but that is unfounded. Poker is an important area of AI research because of bluffing (detection and when to do it). CMU is probably the most active in this area. See https://www.cmu.edu/news/stories/archives/2016/february/poke... and http://www.cs.cmu.edu/~noamb/poker.html. Their bot (Claudico) was demoed at NIPS15.
Interestingly, I've seen solid progress on 3, 6, and 8 in the last 2 months of 2016 and on 7 in 2017 already.
Biggest problems are usually not winning heads-up but meaningfully sampling the state space with more than a handful of players + accounting for the "no limit" part.
[0] https://en.wikipedia.org/wiki/World_Series_of_Poker#Main_Eve...
Role Playing Games would explode if this happened. The ability to generate speech on the fly would create dramatically more immersive worlds.
2016-12-14 => 14 December 2016
Using symbolic techniques, synthesizing algorithms is tractable. See Sec5.3 on pg11- of http://saurabh-srivastava.com/pubs/popl10-synthesis.pdf - Figure 2(b) is the synthesized selection sort algorithm, and Table 3 has QuickSort synthesis time at 160seconds. Very old work (2010) and used templates to guide synthesis. SAT solvers have improved, and much more CPU cycles are available, so in 2016, exhaustively exploring the template space will be trivial.
I.e., rather than reconstructing code from concrete program execution traces, giving the synthesizer (AI?) the ability to manipulate symbolic spaces leads to a tractable search space. Finding specification-matching programs within this symbolic space is doable.
'Beat tier one teams at least 50% of the time in Dota 2 across multiple patches'
It's cool that people are starting to work on Starcraft 2 AIs but the game is kind of dead so competition at the top isn't going to be as strong as it is in more popular games. Also I think Dota presents a much more interesting and more difficult challenge for AI makers.
Not too sure what you mean by 'rushes can be punitive' but Dota certainly has a variety of early game aggression strats which tend to be high risk / reward and are seen fairly regularly.
On the contrary, it is, and the _need_ for such quantities of data is one of the biggest challenges in neural networks.
How about 50,000 games + access to manuals ;)
We also know it uses a Deep Learning algorithm to imitate play from the best humans. And presumably, being stronger than any human now, it could also recursively refine and train on its own games.
Do we know how important the Deep Learning component is? Would AlphaGo be just as strong or nearly so without that part?
I know that over the last few years chess engines have become dramatically more powerful, even on the same hardware, to the tune of hundreds of Elo points. They still stick to the classical techniques though (Alpha-Beta pruning, minimax is still the core). This is not to understate the refinements, but the general ideas are (I think?) only applicable to turn based games with finite moves and a binary mathematical criterion for victory.
Is it possible that the improvements AlphaGo has made are mostly of this type, and the Deep Learning parts are not that important to its actual play strength? Or can we totally rule this out? Is there an in depth discussion of this question somewhere?
But it is an interesting and throught provoking list. And makes me consider more quantitive metrics for qualatitive things like how good a piece of music is.
Imho GI "will arrive" after the point that we have swarms of similar close-to-human-level ai-agents trying to model each other's behavior while competing and collaborating in artificial and natural environments.
All theories of self-consciousness that make some sense imho are about agents trying to model self-similar agents and then somewhow reframing this ability to model past and future self. So you could have self-conscious dog-level agents (yeah, I believe "self conscience" is quantitative and a dog can have some of it and a human a different level of it, not 0 or 1), or non-self conscious above-human-"general"-AIs (though I don't know if you'd want to label "general" any non-self conscious AI, and I think any non-self-conscious above-human AI would get "self conscience" quite fast if you were to push it be be "general purpose" - striving to solve more and more types of problems, you inevitably arrive to some formulation of the "model something like yourself in order to predict its actions" problem).
I'm pretty sure that if you find a way to extract a mathematical model of a human's personality, and a way to transform it to reduce "chemical noise quircks" and other hardware dependent but unnecessary "features"/bugs, you could run it just fine on a big datacenter and have a working immortal human mind.
(I'm also pessimistic enough to believe that we'll solve the "general AI" problem before solving the "mind extraction" one unfortunately. Hopefully our creations don't destroy us completely before solving problem (2) and we have time to pass on some direct immortal legacy of what it means to be "human"...)
P.S. but yeah, I agree that engineering better suited hardware might be actually easier than getting the current type of hardware fast enough or than inventing the mathematical transforms needed to run a mind's set of equations on current type hardware... we're slowly getting to custom hardware for ai ( https://cloudplatform.googleblog.com/2016/05/Google-supercha... ), but I imagine the future will be more analog/clockless...
Citation needed.
> The only proof of consciousness we have is that a brain is required
I think I get what you mean, but that's going a bit too far. For example, in 1950 we could have said "consciousness requires a planet within 100 miles", then the space race would have proved us wrong: being close to a planet isn't required, it's just a coincidence, likely due to life and evolution being more likely to spontaneously occur on planets than off planets.
I would say a similar thing about brains: the fact that up until now we've only seen consciousness in brains is a coincidence, likely due to complex behavioural circuitry being more likely to spontaneously occur in meat (brains) than rocks (chips).
Regarding computability, current scientific understanding goes something like the following:
- The universe, and everything inside it, seems to behave according to some sort of mathematical laws
- The known laws of physics are definitely incomplete (e.g. we have no quantum theory of gravity, we don't have enough experimental evidence to choose between theories of dark matter and dark energy, etc.)
- The known laws of physics seem to be a very good approximation in almost all 'non-extreme' circumstances (this is why we must build giant underground atom smashers if we're to find any discrepancies, and we still don't find any)
- The known laws of physics are computable, i.e. we can use a turing machine to implement the known laws of physics.
- Consciousness arises inside small, warm sacks of meat.
- Small, warm sacks of meat are a 'non-extreme' circumstance.
- Therefore, it seems highly likely that turing machines are sufficient to implement consciousness
This is a non-constructive argument, it doesn't give us an example of a conscious turing machine; yet it's not much different than, say, arguing that collections of atoms can form living creatures, without giving an exact definition of "life" or a receipe for constructing a living creature out of a collection of atoms.
This argument does make some assumptions, e.g. where I've said "seems to be" rather than "is"; these are areas where more research could help, but it seems unlikely that they'll yield refutations to the above argument. The burden of proof is on those arguing that consciousness is unlike anything we've ever studied.
Such a refutation would have to:
- Find new fundamental physics
- Show that this new physics is incomputable, unlike all previous theories
- Calculate a prediction from this new theory (quite tricky, since it's incomputable!), which contradicts predictions of existing theories
- Test the prediction experimentally
This would refute the "turing machines can implement known physics" part, but the argument might still be plausible regarding "non-extreme circumstances" (e.g. if your prediction involves black holes or other exotic situations).
To refute the argument convincingly, you'd need to demonstrate that your previously unknown effect can have macroscopic consequences at ambient temperatures, pressures, in a weak gravitational field, etc. and hence may occur in the brain.
Bonus points if you can demonstrate that your previously unknown, macroscopically-relevant, unlike-anything-else-in-science phenomenon has been routinely occuring inside pregnant women for millions of years (at least) without anyone noticing :)
But evolution started from the smallest thing possible/in existence; Atom, quark, whatever, and built up into the single cell, then multi-cellular and so on. It also existed in a physical world where forces are constantly applied to everything it does. Opposed to a virtual world which is basically a vacuum.
Given a few billion years and this hardware is now known as the human body, built up of trillions of bacteria, undecipherable DNA and countless genes, protein folding and so on. So yes, there is a ton of stuff going on we don't understand. The amount of variables are incalculable.
To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre. At best we can create an approximation of life, something that mimics it but isn't truly conscious.
Consciousness is as real as gravity, we can't definitely point to what they are, but they absolutely do exist.
I don't understand this at all. I have no idea what is meant by "a virtual world which is basically a vacuum". As for "a physical world where forces are constantly applied to everything it does", how does this contradict anything about computability? Turing machines can compute worlds "where forces are constantly applied to everything"...
Are you talking about the sort of naive, approximate, numerical simulations used by scientists to model things like protein folding or galaxy formation? If so, those aren't actually implementing the known laws of physics, they're just convenient, somewhat-efficient approximations. By analogy, just because display machinery used by humans in the early 21st century just-so-happens to be based on a grid of pixels for convenience and efficiency, that doesn't preclude the existence of computations involving other representations, like vectors with unbounded precision (or, if you prefer, busy-beaver-bounded precision).
> there is a ton of stuff going on we don't understand.
This sounds like an argument from ignorance, which would be a fallacy in itself, but in this case we do understand some of what's going on! This "ton of stuff" is 'merely' a consequence of the underlying physics. Whether those consequences are complicated or not doesn't matter at all, because computability has nothing to do with being complicated.
If I run `while true; do python < /dev/urandom; done` my laptop will do "a ton of stuff" that I don't understand, some of which may be quite complicated; but I can say for certain that it will never produce jelly beans out of the USB port, since there is no mechanism for such a thing to occur, regardless of how complicated the signals in the wires are. On the other hand, if there were such a mechanism, it would only take one simple command (`sudo eject /dev/jellybean`) to cause its dispensement; no need for any appeal to complication at all.
Likewise, regardless of how complicated the interactions were in the formation of the Earth, life, our brains, etc. we know that such things must be computable, since there are no mechanisms in the known laws of physics which can lead to incomputability.
You can string together as many computable calculations as you like, far more than the universe could ever have performed in its entire history, and the result will still be computable. That's just how computability works.
In order to have anything that's incomputable, you need something other than computable components (i.e. jellybean-dispensing hardware, rather than more software). It doesn't even need to be complicated at all! If even just something tiny, like the magnetic moment of an electron, were incomputable by a Turing machine, given the laws of physics and history of the universe as input, that would be enough to support your argument. You don't need "tons of stuff", it's makes no difference. Yet even that seemingly tiny quirk would require that we throw away the known laws of physics. It seems unlikely.
> The amount of variables are incalculable.
Do you mean this as a linguistic gesture, or in a technical sense? For the latter, I would say it's maybe impractical to calculate, but certainly finite. We could upper-bound it by, say, Graham's number.
> To think we can replicate all this with some nano scale transistors, and mathematical approximations, is just bizarre.
I never said that, and as far as I can tell neither did any others in this thread.
Firstly, I would think of such claims as hubris. That such engineering feats could be achieved by a planetsworth of apes after only a few million years would be very surprising, and I certainly wouldn't want to bet on it happening either way before we become extinct.
Secondly, I don't understand what you mean by "all of this"; are you claiming that the entire history of the Earth must be reproduced before a conscious being can be made? In that case, what do you think is happening inside testicles, ovaries and wombs that can manage "all of this" in nine months?
Regardless, the arguments I made, and that of user nnq, were discussing computability. Computability (AKA universal computation) is a mathematical phenomenon regarding what can and cannot be computed by particular mathematical systems. The reason it's called universal computation is that, if something can be computed by one universal computer (e.g. the universe, a brain, a laptop, etc.) then it can be computed by all universal computers (e.g. the universe, a brain, a laptop, etc.).
This applies to all computations, whether we know what they are or not. Nobody (yet? ever?) knows what computation is being performed by some particular brain which gives rise to the phenomenon we call "consciousness", but we don't need to in order to know that that exact same computation can also be carried out by any other computer (e.g. a laptop).
No need for approximation. No need for appeals to ignorance. No need for appeals to "emergence". No need to even know what it is that brains do (since it must be a subset of what the laws of physics do, and the known laws of physics are computable).
But, as I said earlier, this isn't a constructive argument: it doesn't give us an x86 binary we can run. Also, as nnq pointed out, even if we did have such a binary, there's no reason to think that it might be in any way practical; it may take many trillions of times the age of the universe before it finishes calculating a single neuron activation. Such things don't matter at all for computability, but they're inconvenient for us mere mortals.
Huh? We can't even translate texts correctly between mainstream languages yet.
Or something like that anyway. It's not well explained.
If these are "well specified", then the author and my boss are operating on some definition unknown to me.
Also, quite a few tasks don't seem to be obviously about AI. Stuff like:
> Play poker well enough to win the World Series of Poker.
It's one thing to play Starcraft or Go well. But implementing this is either mostly a question of luck, or an incredibly lifelike robot with incredible facial recognition skills. Very different tasks!
> Beat the fastest human runners in a 5 kilometer race through city streets using a bipedal robot body.
This requires no AI, just a lot of improvement in energy density and actuator technology.
In terms of strategy though, it is the next level up I'd say. Videogames come first, then more difficult games such as Poker come later where you need to be able to tell a bluff from the truth. Much more subtle.
"This requires no AI, just a lot of improvement in energy density and actuator technology."
It did say city streets though, which makes me think it would need to strategise and be able to:
1. Adjust speed and take corners close such as what you see in F1 races.
2. Switch between endurance and sprinting when necessary to maintain energy for the whole race.
3. Run behind others to stay in the tailwind and reduce energy cost.
All require human level sensory input and a strategising component.
Drag is quadratic with respect to speed, so switching between endurance and sprinting is sub-optimal. Just maintain pace.
And at these speeds and the required clearances for safety, the benefit from drafting is less than 5%. This is huge for runners pushing the limits of human performance, but if a robot has the same limits where this matters, it will probably be by design.
This would require real-time object detection and avoidance, following signs/social conventions/etc., complex route-planning, avoid people/cars/etc., predicting the movements of others, predicting how others will predict your predictions, etc.