When he says something like "600 milliseconds is not a lot of time", what is the limiting factor? Is it the sampling rate of sensor input (video)?
It would be seriously cool to use the speed of motion of the backboard to get extra control and, e.g., need to move it less.
But that would take a hard real-time system, and be correspondingly more fun.