I think one major draw to human-like for factors is the reuse of existing ecosystems and tools. If you have human-like grasping, you can reuse tools and utensils for human hands, otherwise, you need custom attachments. If you have human-like legs you can navigate stairs, wear pants for customization, and possibly operate a car or bike.

Its a bit like choosing JS / python -- of course performance is inferior to a compiled language with highly tailored code, but they are flexible and have an ecosystem that might do 99% of the lifting for you.

But in isolation, I agree with your idea that specialized robots with form fitted specifically to task will likely outperform a more generalized solution in a specific domain of behavior, the more generalized will likely outperform in flexibility and reusability (e.g. capable of reusing the human ecosystem).