Reinforcement Learning: An Introduction

In theory yes you could. RL is based on trial and error. The agent senses the environment in the form of states (ligh=state1, dark=state2) and it performs actions according to a policy to receive positive or negative rewards. For example your goal is to have a room always lighten, the agent will sense the environment (it is dark) it will turn the lights on and it will receive a positive reward. Eventually the agent will learn that in order to receive positive rewards it needs to turn on the light when it is dark and turn it off when it is light. Actually RL it is more complex than that but that is the basic idea. If you are interested go to scholar.google.com and search for papers about reinforcement learning. Important authors are Barto, Sutton, Watkins (Q-Learning), Littman, Kudenko, Stone, Clauss, etc.

Could I train a robotic hand to catch a ball with this? What would be involved?

This has been here before : http://news.ycombinator.com/item?id=795388