An AI First World
avc.com
avc.com
(I also wish it said more, but his blog comments tend to have better discussions than even hn, sometimes, so maybe he is just trying to kick that off. I wish it were a better technology for commenting, though.)
Still, the "single-issue commenter", as exemplified by both of them, can be valuable to a good community. @michaelochurch's writings have definitely shaped my thoughts, so kudos for him, and, if I were in the position to use it, @graycat's bibliographies would be rewarding to mine (unless his knowledge is outdated, as @barrkel suggests). Yet I can't help but get the impression that the single-issue commenter is very prone to overexposing themself, especially if a large proportion of their comments are multi-thousand word diatribes that recycle most of the same themes. Not to mention, people readily pattern-match prolix commenters to complete crackpots, even though both those users seem to be above that level. My impression is that single-issue commenters would be best served by posting a lot for a year or so, followed by some time to lie low -- if their contributions to the discussion around their issue are really valuable, their posts should stay in moderate circulation as other users refer to them in later discussion.
Also _not_ clarifying that we are not dealing with any sort of hard AI (which is what the layman understands by AI) is always a bad sign.
All in all just a marketing piece.
AI is today's "one word: plastics!" (https://www.youtube.com/watch?v=Dug-G9xVdVs). Much hype, some substance, lots of people throwing money around.
Here is a good article that goes over the topic: http://spectrum.ieee.org/automaton/robotics/artificial-intel...
That's quite far from the truth. You could say that they are now at the point where they are able to respond to direct, simple questions/commands.
I am not saying that we haven't seen some great advances in machine learning and language algorithms, but they are still a far cry from anything I'd call intelligent. NLP is still rather primitive (although impressive compared to where we were two decades ago), and bees are able to navigate better than self-driving cars or drones, yet I'm not sure they would or should be called intelligent.
I have no problem with using the term AI for current technology -- which is certainly cool and useful -- as long as we understand that the "I" is little more than a marketing gimmick. Otherwise people may think that these algorithms are the first step towards "true" intelligence -- i.e. that true intelligence is some version N.0 of those algorithms, that it is a complicated yet natural extension of them -- and at this point the knowledge we have to say whether this is true or not is pretty much zero.
No, it is not. Because it's in stricter terms (like character or picture recognition, for example)
For example, face recognition performance is often measured by comparing two random faces taken from a sample set. In real life, the "problem" is almost always to match a random face against the entire set of faces you remember. Performance in these two tasks is not at all equivalent.
I've seen similarly weird problem formulations in many other AI research papers.
But, very recently, some interesting hardware, software, data sources, and business models have evolved and converged that make 'AI' much more practical and realistic. And, just take a look at recent research papers and popular open sourced projects, and it's clear that many of the smartest people in the industry are working on it. Some very creative and innovative things have been happening in an ambitious space that had been kind of stagnatant for decades.
Think of AI of today like 'The Internet' was like like 1991. Probably more impactful, actually, because the major tech companies are throwing billions at it. There are many more people with skills and knowledge to move it forward. And, we all know that Alphabet is moving toward constructing personal intelligent agents for everyone on the planet. It's going to be a very interesting decade.
IMHO, involvement of major corporations throwing billions at it is an anti-signal for the success of a technology. What'll really make AI take off is effective 80/20 solutions combined with novel applications to fields that people consider dead. I have no idea what those solutions or those fields are - I'd be a lot richer if I did - but I'll predict that we won't call them "intelligence" when they arise.
Also, we already had government-backed standard networking with OSI. The US, UK and EU governments were big OSI backers and didn't want TCP/IP. (2)
However, 20/20 hindsight is a wonderful thing ;-)
(1) https://en.wikipedia.org/wiki/AI_winter (2) http://spectrum.ieee.org/computing/networks/osi-the-internet...
What I'm looking for in the current AI hype is profit. Will IBM be successful in making Watson turn a profit? Will Google leverage alphaGo technology to not just win games, but also make profit? A big one here is self-driving cars: if the AI becomes good enough to sell to the public, that will be a huge market for AI.
So that's what I'm looking for. If the ideas become profitable, then AI will not enter into another winter.
Edinburgh University, under Michie, was leading the world in A.I. Michie, Ambler & Popplestone's 1969 Freddy Robot could assemble scattered bricks using A.I. vision & GOF symbolic AI. Michie (& Chambers) BOXES algorithm was succesfully learning to balance poles using reinforcement learning, from raw-sensor data outputting motor torques, real robotics trained end-to-end not simulated. Edinburgh had a rival LISP known as POPLOG (Popplestone & Burstal) used UK & Europe wide.
BOXES was akin to todays Neural Nets. Geoff Hinton attended Edinburgh briefly in the aftermath. What might have been ?
Edinburgh had both symbolic A.I. planning, a working vision system & pragmatic bottom up machine learning until '73 & Lighthill. Edinburgh's was a unique approach - with bigger budgets the American labs favoured simulation (until 1985 when Rod Brooks showed it was a dead end) and without BOXES, Freddy became RAPT, symbolic, programmed GOFAI & no learning. RAPT failed because top-down reasoning was a computational quagmire in the real world & it was too fragile to the slightest decalibration.
Edinburgh's hybrid GOFAI & behavior-based approaches got round calibration issues with SOMASS, Eddie & Push-Slide. But programmed solutions lacked the crucial element of learning - Michie's BOXES.
Concurrently, Michie, exiled to Glasgow & private industry, found great success helping the Japanese revolutionise the assembly line & in the US with automated factory control & satellite piloting for NASA. All reinforcement learnt using BOXES but he & AI were pariah in the UK as he laments in 1987:
[Michie from 18:24] http://www.bbc.co.uk/iplayer/episode/p0306rt1/micro-live-ser...
Michie's nascent hybrid of raw sensor machine learning & GOFAI embodied in robots is just being rediscovered in Abeel & Levines work today. http://arxiv.org/abs/1504.00702
An AI winter is a terrible thing. Progress lost decades. Tragically the 2002 fire may have destroyed Michie's undigitised archive.
We'll continue laughing and shaking heads over AI tech, ruing the hype, then in 30 years everything digital will be made out of it.
AI is useful because it changes the economics - it either enables the removal of humans from an activity, or makes economically feasible an activity that would otherwise be unfeasible at scale due to the need for humans.
I think that applying the label "AI" to some algorithms only serves to give us a very impartial sense of awe towards them that goes far beyond what their actual performance merits.
But all algorithms do what humans used to do. So is AI the name for algorithms that do the work humans (in Western societies) still do, or did until recently?
> human command will move to higher and higher levels of abstraction
So am I on a different level of abstraction when a human is driving my taxi than when a computer does it? ;)
But your former driver certainly is.
Neural nets and learning algorithms that are trained to do things are different and can cope with decision spaces involving a numerous variables where deterministically working out all the possible values and corresponding expected outputs isn't viable.
Yes, it's just that there's nothing necessarily intelligent about learning through statistical clustering. I mean humans don't learn something by looking at many thousands of examples; usually a couple suffices. Maybe that's just a higher-level statistical induction that yields intelligent deduction[1], but we have no idea. We know that statistical clustering, at least in biological brains, is a more primitive computation than, say, bubble-sort, but we don't know if it is necessarily the primitive building block for an "intelligent" algorithm. It's also true that we don't have a good definition of what intelligence is, but we do have a probably-reasonable intuition about it. Now, don't get me wrong, I understand why people feel like this should be called AI (for various reasons, both backward- and forward-looking), but it's important to remember that we have no idea where these algorithms are placed on the spectrum between bubble-sort and human intelligence.
If anything, machine learning is more of a testament to how effective statistical algorithms can be (which is either "very" or "hardly", depending on the problem and the problem) without doing anything that could be called intelligent. This is not too surprising. Insects are also extremely effective -- and adaptive -- at some tasks without being intelligent at all.
[1]: Word embedding (vector representation of words) is probably the closest we've ever got to deduction, but we don't know that this model generalizes well.
theres not much to be gained from arguing about the definition of intelligence, but however we use the term I think the above is what distinguishes these systems. What term would you prefer to use for them?
Inductive algorithms, statistical algorithms, learning algorithms.
Any algorithm for solving practical problems embodies a model of the problem domain. Traditionally we try to bake in as much of our own experience as possible in the model/algorithm.
In some fields, we're now seeing for the first time that it's better to not do that, to make fewer assumptions, build in less of our own knowledge, and just use (new, powerful) general purpose learning algorithms. With enough data, they can make sense of it themselves - our attempts at explaining it through model-building just get in the way.
Anyway, statistical algorithms, adaptive algorithms or inductive algorithms would be much better, less misleading and less breathless -- if less inspiring -- names for those programs than "AI".
Before, using machine learning to build models from the data was in some sense a last resort, for things you couldn't get a grip on modelling by hand from your own understanding. When you could, you preferred hand-made models because they did better.
To take a concrete example, the top computer go program before Alphago, CrazyStone, still had hand-crafted patterns to prioritize which moves to explore during playouts. The author (and even more his competitors) tried to find better patterns with machine learning but they couldn't. The weights of the patterns were tuned with machine learning, but any attempt to e.g. evolve patterns with genetic algorithms just did worse.
Now, it's going the other direction, we're seeing very general models only constrained by training data beat carefully hand-crafted models in area after area. And models are becoming more similar, more generalized as well - many people have noticed that state of the art recurrent and feedforward nets increasingly look more like each other.
I disagree that it's a far cry from intelligence. I think people's ideas of intelligence are fuzzy and inconsistent, and it's far too often confused with things we associate with intelligence (from consciousness to independence and arrogance).
I can certainly agree with that -- it's not clear at all what intelligence is -- but induction and pattern recognition (really, statistical clustering) are either not sufficient for anything which we'd reasonably call intelligence, or can be found in creatures that we'd say have negligible intelligence (insects). Again, I'm not saying the achievements aren't impressive or that they don't resemble anything we can associate with biological beings, but even most biological beings aren't intelligent.
In any event, no one claims (with any evidence) that the current "statistical induction" algorithms (inspired in part by some features of biological systems) are a simple, primitive form of "general intelligence" algorithms. They may well be, but we don't know that. So the "I" in AI is, if not a marketing gimmick, at most inspirational and aspirational. As far as we know, those algorithms are as far from human intelligence as solid-fuel rockets are from warp drives. Understanding it otherwise only leads to people conjuring up fantastical deadlines to achieving "true" intelligence. For all we know, we are as likely a century away from "true" AI as we are five years away.
But who do the AIs work for? Google, Facebook, and Homeland Security?
To expand upon my comment: For most programmers Decision Trees & Nearest Neighbor, at a first glance look very promising and seem like natural solutions to lots of problems. However in reality one can easily prove that Decision trees tend to be suboptimal even with trivial problems due to the brittle nature of the rules represented. Also Nearest Neighbors methods face problems due to difficulty in deciding optimal space and metric (Euclidean distance is not inherently better). The reason why "Deep Learning" methods trained using non-convex optimization perform so well is because they can represent a hierarchy of features at different granularities. They can be "designed" to favor one task over the other. E.g. by choosing correct loss (triplet loss), network structure, etc.
If you are interested beyond Image recognition problems, I highly recommend reading on interesting approaches such as Triplet loss for Face Recognition , Semantic Segmentation, Attention based models and Unsupervised Visual Representation Learning by Context Prediction.
An interesting outcome of Deep Learning research is that a lot of papers are extremely accessible and involve little proofs/difficult math. E.g. This is a good one https://www.youtube.com/watch?v=dUgRR-JFE8s & http://web.cs.hacettepe.edu.tr/~aykut/classes/spring2016/bil...
But from decision trees you get decision forests, which can't be said of bubblesort
Perhaps one of us should write a classifier to see if we can predict which will be best for a problem a priori.
Hinton has recommended ensembling diverse solutions (for Kaggle) & including random forests.
The adage "The truth is somewhere inbetween." or wisdom-of-ensembles is profound in this domain.
There's (at least) one thing that could change this, which is that the machines get so complex that we can't really grok the details any more. You could argue that this is beginning to happen in Deep Learning, where it can be hard for humans to map the functions being performed later in the network to anything with intuitive semantics relative to the problem.
Perhaps in retrospect we will consider the Singularity not a spectacular thing where the machines suddenly got super smart, but a more mundane thing where most of community gave up trying to understand the machines and just use them ever after as black boxes.
http://avc.com/2016/04/an-ai-first-world/#comment-2640180803
In the short term, perhaps, at least for mobile devices. But there's a lot more bandwidth in visual input. So I suspect visual input via EMF. Eventually thought.
Strong AI would completely change the power structure of human civilization. It would hold all the power, much like States try to do today, and it would dole out little bits of it to the puny mortals so they can do what it wants them to do.
Eventually I think human beings will come to worship strong AI. It will become a God of sorts to them. Like religiously because AI provides what the real/imaginary God never could.