Ask YC: AI
The field of artificial intelligence has been always something I wanted to take a peek on, but never really had the chance. Could people here recommend me some good books to start my journey on this field?
I'm assuming you mean "AI" in the sense of "fancy mathematical tricks to solve domain specific problems", not in the sense of artificial general intelligence.
Norvig also has some great essays and example AI code on his site, http://norvig.com/
1. Classification and Learning (aka is that apple ripe? or what's the best website for a given search?)
2. Modeling / Machine vision (Where are the rocks around this rover.)
3. Goal Seeking (AKA what's the best path from here to DC.)
With enough resources we can do any of the above fairly well. So using AI is more about understanding how to link the above activities to some useful problem. AKA control a rover when your ping is 15 min or solve a CAPTCHA.
I like out of the box solutions more:
It's one of the best books on AI and it's a "standard" AI textbook for most of the intro AI courses.
It's also great as a reference (I have it on my bookshelf beside me) and you can quickly look up algos and implement them in your fav language.
Also, google "Andrew Moore CMU" his homepage is full of useful tutorials
That said, "AI" is too vague. What do you want to be able to do?
Keep in mind that AI is a broad field. As such you might find it more helpful to identify which computer intelligence problem(s) you want to solve, then start accessing the AI writings that discuss how to solve your problem. This won't give you a general overview, but it at least would reduce what you'd need to learn about to produce something useful.
is a great book, and a nice complement to Norvig and Russell, which is somewhat more theory-oriented.
The classical AI stuff is still interesting, of course, and still has practical applications (ie A* search for games). But I tend to count "the other stuff" (ie Data Mining) as AI, too.
Perhaps similar, and available as a free ebook: McKay, Information Theory, Inference Theory and Learning Algorithms, http://www.inference.phy.cam.ac.uk/mackay/itprnn/book.html
It is ironic that the man who wrote the book uses different methods (maths) to extract meaning from data at google ~ http://news.ycombinator.com/item?id=90298
Something smells about AI.
It has been well funded, attracted the best minds (Minsky, McCarthy, Newell) at places like MIT, Stanford and resulted in lots of useful languages & tools. [0] From Cyc to Powerset companies have tried to use these techniques trying to assign human like intelligence to machines yet have failed.
Why?
Bruce Stirling reckons it has to do a lot with the language. [1] Assigning intelligence to what is essentially a 21'st century equivalent of knitting mills or steam engines of yesteryear. Maybe thats why Google has taken the approach it has. Instead of creating intelligent software to find the meaning, take an insight into users who have pre-assigned intelligence to documents. And use well known sorting, linking techniques and software engineering along with not-so-well applied math techniques. Thus make google appear to make sense of what we search for.
[0] http://en.wikipedia.org/wiki/Artificial_intelligence
[1] ITConversations, 'Bruce Stirling, The Internet of Things, 58m, 26.4 mb, 2006MAR06' ~ http://www.itconversations.com/shows/detail717.html
Our genetic code isn't that huge, but the amount of information contained in our brains is. All that information must be coming for somewhere. I refuse to believe that evolution has managed to create _that_ awesome compression algorithms.
Maybe this is what Google does, in a way. They happen to have a huge corpus of machine-readable data, which is in a sense equivalent to what a sophisticated sensory apparatus would give you. I intended to disagree with you when I started writing this comment, but now I'm not so sure. Human intelligence obviously doesn't spend its time reading petabytes of information on other people's search habits, but maybe Google is actually onto something.
Reading through the Russell & Norvig book would give you a good overview of the sorts of techniques available.
I see lots of specific solutions to specific problems. That, to me, is the best approximation of intelligence. To the extent that many specific solutions can be approximated and optimized, we'll be very well off. It's easy to denigrate a calculator and spell-checker, but I seem much more intelligent than I would be without them.
Logical Fallacy Alert! http://en.wikipedia.org/wiki/Argument_from_ignorance
I don't know how to build a rocket ship nor to do a heart transplant, but I'm fairly confident that these are both feasible.
Moreover, you prove your own point - the one I was agreeing with initially out of everything said in this thread. A rocket ship or heart transplant are specific solutions to specific problems. The analogy to intelligence would be one universal solution to flight or one universal solution to getting oxygen and glucose into cells. It's not that a solution isn't possible. It's just that the hope for THE solution is misguided (see also THE cure for cancer).
I really like the flight analogy because of the length of time people put into finding THE solution. But what we end up with instead are rockets, and prop planes, and jet engines, and airfoils, and gliders, and helicopters, and hoverboards...
For more philosophy, read Minsky's books / papers, and the readings outlined here: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Minsky's paper "Steps toward Artificial Intelligence" reads like a summary of 6.034 at MIT.
Speaking of which: http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Compute...
Any suggestions on where to get more of that higher-level commentary?
Doing something original might actually be, you know, original.
If you follow the classical path, you will become bogged down in the vast field of AI.
Ramanujan started from first principles and worked from there. My guess is that the person(s) who create "actual" AI will use that approach.
But there's value in studying things that didn't work. A huge number of obvious approaches to AI have been tried already, and the insights on why they don't work, or why they work on this problem but not this other one, are often the result of huge quantities of time spent on subtle traps and dead ends. To ignore them risks wasting your time all over again.
I think AI boils down to just the problem of managing complexity that all of cs is about. The eventual solution will be vast, and will require designing lots of separate subsystems that work in very different ways and yet need to communicate in subtle ways.
That emphasis on scale goes for research in general, I think. I recommend this video of Malcolm Gladwell talking about the nature of genius and how it's changed over the years. http://www.newyorker.com/online/video/conference/2007/gladwe... He compares the decoding of the rosetta stone and linear B 40 years ago with Andrew Wiles proving Fermat's last theorem in the past decade, and how fundamentally different their respective approaches are.
Both approaches may work, but you have to decide what attitude you want to take. You can either try to be a single monster-mind like Ramanujam sweeping through vast areas of research, or you can assume that won't happen and resign yourself to a lot of effort and learning before you're able to synthesize something useful.
To summarize: I agree that you want to avoid cluttering your mind with the ideology of past approaches. There's huge value though in simply studying the episodic history of a field, to be aware of what has been tried, what worked and what didn't.
In fact, reading what somebody did will often help you avoid going down a path. I've read of many many more algorithms, successful and unsuccessful, than I've ever implemented.
Yup. We've already established that one has to prioritize what to implement. The priority clearly shouldn't always go towards doing something rather than reading what somebody else did. (Children would never learn to write that way.) Instead we prioritize differently at different times, and we try to have lots of feedback between reading and doing, by things like active reading.
I'm not saying that you have to learn everything before doing anything. That's a strawman. I'm saying you shouldn't ignore any source of knowledge, that you should look everywhere taking the good and leaving out the bad. And that's a Bruce Lee quote as well.
The concern about contagion is valid; my attitude towards it is that it happens not so much from what you read but who you talk to and the social circles you move in. Keep those diverse and you should be fine.
If you don't learn everything, then you must have some independent criterion with which to pick and choose; and having such a facility, one may as well simply start down whichever path one feels is most appropriate anyway.
In other words, if you can somehow rule out large swaths of possibilities without even trying them, then you must have some intuitive feel for the problem (or at least, have an intuitive feel for what you think the solution is likely to resemble, whether or not you turn out to be correct). In such a case, you are investigating based on your own intimations and your efforts will develop from there, rather than from from a top-down view -- which has so far been amply demonstrated not to work.
Edit: The recommendation to look "everywhere" is untenable, because at some point you have to stop casting about if you are going to get anything done yourself.
The thing that is independent/unique about somebody isn't a static criterion that remains unchanging over time, it's more the trajectory you end up taking that's constantly evolving over time. All I'm saying is: don't exclude reading about past failures when you make your choices on what to do next. It's fine to say ok, let's stop reading for a while and try building this one thing my mind has settled on. That's a great habit to acquire.
The unpronounceable author of Flow: http://www.amazon.com/Flow-Psychology-Experience-Mihaly-Csik... says the most successful people alternate periods in closed states when they focus on a specific concrete project with periods in open states where they demonstrate huge appetite for new ideas.
I'm starting to think we're saying the same thing in different words and with slightly different emphasis. It is thus with most interesting conversations.
I LIKE TURTLES
Whenever I see someone succeed rapidly I assume it isn't a newcomer, that there are depths of effort I cannot yet see.
I don't think a beginner is going to create AI by "luck". But a beginner's mind is not cluttered in the same way as an expert's. This may be what it takes to get working AI.
Expert knowledge is appropriate when discussing proven solutions. AI is speculative at this point.
You don't know how many times I have heard a student claim that they have made some type of major advance. But on inspection it usually comes down to something that requires exhaustive search.
I think that there is definitely a place for new ideas in the field, but I don't think they will come from complete novices. I think they will come from those who have CRITICALLY read the basic literature. And probably from those who have read more than just the basics too.
My own pet peeve on the field now is the over emphasis on asymptotic complexity. Sure its important, but its not exactly an AI result. The scruffies need to make a come back and start coding! See my own meager efforts (http://www.anaphoric.com)
It might even be something simple, overlooked, done a little differently.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
I can't think of any phenomenon less likely area to have a simple algorithm than intelligence.
Cf. Evolution.
You're darn well gonna wait -- and LIKE it!
Fortunately, it takes significantly less time than that -- no more than three and a half billion years. Haha, and you were worried!
It's actually amazing if not ridiculous that researchers always start from the other end of a cigar: they raise ideas, one after another, but they don't even try to explain what problem exactly they are solving.
The process of discovery is hands-on, learning by doing. You don't know all the rules at the outset, or all the definitions, or even know what you don't know. You find out as you go along. Thus it has been for every great invention from sex to agriculture to post-it notes to landing a man on the Moon and back to on-demand porn.
The real requirement is that we have some way to test it; that we can do, e.g. by conversation. To cop an example from quantum mechanics (which I know nothing about), we don't have to understand why it is like that in order to make predictions with it.
We don't need a clear and unambiguous definition of intelligence in order to tell that other human beings are intelligent. Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
This was quite impressive, and on the right track, seemingly: http://hci.stanford.edu/~winograd/shrdlu/
The problem here is that not every human being is intelligent and yes, we are actually trying to define intelligence through IQ tests, for example.
Likewise, we don't need one in order to start trying to create an artifice that impresses us into thinking it also is intelligent.
So when you need accounting software you say: "write me something that will be as clever as my accountant". Is that the way you formulate tasks for software engineers?
Fail. IQ tests only measure how well you do on IQ tests.
> The problem here is that not every human being is intelligent
Unless they're in a vegetative state, they are more intelligent than any artificial system, so far.
> Is that the way you formulate tasks for software engineers?
The only way to make a specification so precise that it does exactly what you want is to implement it, and then the code itself becomes the specification.
And yes, it's the way programming works - when you know what you are trying to achieve. Software is about input and output and unless they are deterministic, you can't write code.
Obviously people say that all the time despite our lack of a definition for exactly what it is, so we apparently don't need an exact definition.
> and unless they are deterministic, you can't write code.
You might be interested to know that the behavior of your OS isn't deterministic once you add a second CPU.
SHRDLU is nothing more than a really well-done ELIZA for a really small domain.
That said, I also don't think you need to be an expert neural network wrangler and genetic algorithms shaman if you want to study "true AI". You should however have an understanding of the fundamentals behind the techniques used in classical AI so that you can rationally dismiss them, or hell, maybe you'll even find some amazing lead or inspiration in the existing work. After all, the field of AI did start out with ambitions of creating "true AI" (whatever that is).
Of course a real AI would be much better at all those things you mentioned anyway -- like deciding which of my e-mail messages are spam.
Or carrying on a conversation.
P.S. I'm an AI.
Yes, I fully agree that a "real AI" would be able to do those things better, but my point is that we do have real practical utility from the narrow AI tradition, and their work isn't for nothing.
Then you're hopelessly out of touch with what most people mean when they say "AI", and what the person who asked the question wants to learn about ("Learning Algorythms and Decision Making programs").
Hopeless? Doc, are you telling me there's no cure?!?
In fact, when most people talk about AI, they mean it in the sense of, "do you think computers will one day be as intelligent as humans?". Cf. the moviefilm "AI". If that kind of AI is what you wish to pursue, then the so-called field of AI isn't going to get you there.
I'm reclaiming the term AI for actual AI from the minority that's misusing it and we're having a Pride march on the 23rd. Wheel out your servers and "represent".
Now that's an excuse if I ever heard one. "Uhh... we weren't seriously trying... uhh..."
That's also why I'm not dating Natalie Portman!
> Considering that trying to build strong AI is a pipedream anyway (IMHO)
That's hilarious. Heavier-than-air flying machines, a man on the Moon, sending pictures through invisible AIRWAVES?!? Pipe dreams!
Those people have mostly switched to using the term "Artificial General Intelligence" when describing "actual AI".
The rest of the world just says AI. It's like a million to one, we win. Don't make me have to call M.C. Hawking.
Never heard that term before.
And to quote Office Space: "Why should I change my name, he is the one who sucks". So personally I won't adapt "Artificial General Intelligence" into my vocabulary.
Meanwhile humans farm gold and solve CAPTCHAs.