Mario Lives: An AI for creating a living and conversing agent [video]

if you're interested in more of these calls (some are quite funny) there's an entire reddit dedicated to it:

http://www.reddit.com/r/itslenny/

The speech recognition and synthesis are primitive compared to any smartphone assistant these days. The natural language processing is about equivalent to SHRDLU from the 1960s [1]. It turns out that this approach based on manually constructing syntax trees and applying simple logic can make some fun demos but is ultimately a dead end in terms of making systems that are actually useful, as was discovered in the "AI winter".

The part that controls Mario looks similar (if not identical) to this: http://aigamedev.com/open/interviews/mario-ai/

If you want to see the state of the art in using AI to play video games, look no further than "Playing Atari with Deep Reinforcement Learning" [2], where a single general AI system learns to play many different games. The generality is what makes it impressive. Its only inputs are pixels and score, and its only outputs are joystick and button state, just like a human player. This makes it unlike these Mario systems which are hand programmed very specifically for Mario, and use special instrumentation of the game state that skips pixels entirely.

[1] http://en.wikipedia.org/wiki/SHRDLU

[2] http://arxiv.org/abs/1312.5602

This is absolutely an impressive demo, but the way to look at it is as a 'mash-up' of a variety of components that have all been demoed before in isolation and usually with a lot more attention to the technology behind it but a much less slick presentation.

So kudos to the people that put this together but it is mostly the mashup that is new, not the individual pieces and those pieces have demos all their own that make for very interesting reading and viewing (where applicable).

I should do a blog post on this, but that will be a ton of work to properly represent the state of the art (which this demo does not).