A factory is a fully controlled environment. All that neat control goes down the drain when you’re confronted with the outside world—weather, wind, animals, plants, pollen, rubbish, teenagers, dust, daylight, and a myriad of other factors ruining your robot's day.
I'm not sure that "humans will still dominate work performed in uncontrolled environments" leaves much opportunity for the majority of humanity.