If you've ever done residential wiring or plumbing fixes in an older home, I suspect you'd share my opinion.
That's why I differentiated between robots working on new construction sites, where I do think that a 15-20 year timeline is entirely feasible (not necessarily desirable, but feasible). That's an entirely different task than renovations in an existing building, which is much *much* harder even for humans.
Also note that there's another way of thinking about where technology will take housebuilding that already exists: more factory-assembled modular cosntruction. That's less popular in the USA, with its focus on single family "unique" homes, but is taking off elsewhere and is already highly automated.