You say you're not interested in path planning, but path planning and control are really on the same spectrum, which is motion planning. For instance, where to next put big dog's leg is a control problem, but also a path planning problem (how do I get the leg there? Where is the best place to put the leg? Does the leg collide with anything?). If you can't answer these questions and are blindly putting your leg wherever, you're not in control of anything.

Here's a good paper on RL in robotics: http://www.ias.tu-darmstadt.de/uploads/Publications/Kober_IJ...

RL is not widely used for control, but it has yielded some impressive results. In my experience I had a highly dynamic system for which I built a hand-tuned model for motion planning. I also built a RL model and trained it using the hand-tuned model. The RL model performed more than 50% better than my very best efforts.

Also, some of my favorite textbooks:

Principles of Robot Motion by Choset et. al.

Statistical Robotics by Fox, Burgard, and Thrun

Linear Systems Theory by Hespanha