Looking at the advancements in low cost flexible robotics I'm not sure I share that sentiment. Plus the LLM craze is fueling generalist advancement in robotics as well. I'd say we'll see physical labor displacement within a decade tops.
The number of edge cases when you are dealing with physical world is several order of magnitudes higher than when dealing with text only and the spacial reasoning capabilities of the current crop of MLLMs are not nearly as good at it as required. And this doesn't even take in to account that now you are dealing with hardware and hardware is expensive. Expensive enough, that even on the manufacturing lines (a more predictable environment than let's say landscaping) automation sometimes doesn't make economic sense.