> Knowing when you don't know the answer is something that has been lacking

Definitely, and I'm not sure why that space isn't getting more attention. For RL there's KWIK (knows what it knows) http://www.research.rutgers.edu/~lihong/pub/Li08Knows.pdf [pdf]. A large class of problems (the robo TA being one of them) lend themselves very well to AI with error bounds, for lack of a better term.