The GP specified a robot with three joints in his illustrative example. Each joint has three axes (physical dimensions) along which it can move. Each movement has an arbitrary scale of movement of 0-10, since the GP discretized continuous space into 10 buckets arbitrarily for sake of illustration. So 10 choices for each of 3 joints for each of 3 dimensions. That comes out to 10 choices for Joint 1 Dimension 1 x 10 choices for Joint 1 Dimension 2 x … 10 choices for Joint 3 Dimension 3 which comes out to 10^9.
You have 3 joints with 3 dimensions. So 9 variables you can control. If each variable has 10 choices, and you need to make a choice for each, that is 10^9 possible combinations.
"3 joints" -> What are the 3 dimensions that joints move along? I naively would have thought one or two (ie. how much spin on one axis and how much spin on the other axis) just by naively looking at my elbow for a second.
For an arm of fixed length, polar coordinates have 2 dof.
Yes, there are multiple types of joints. In this case, you could think of a ball joint like a shoulder or hip. They move "up-down" and "left-right" but also rotate.
I presume it's 3 dimensions for each joint, so 10^3 * 10^3 * 10^3
I think the problem is exaggerated. Even with three ball joints, the action space is not that large since there are constraints on the velocity of the joints. They have to move gradually. So the actual action space is a lot smaller. A lot RL problem has similar continuity constraints, cuz in real world we are dealing with time series signals. I am not a expert in the RL domain yet, so I open myself to any opinion.