Efficiency of solar or wind doesn't have to come exclusively from improved conversion efficiency - that is from taking advantage of all the photons and all the wind. Efficiency gains also come from using less material for the same harvesting area. He is likely right that once you're in the 50% efficiency range, you can't get 10x conversion improvements. And when you are already at tens of micron thicknesses in the key materials, you can't expect to go much further.
This is not the case for nuclear where energy density is not the problem at all, and semiconductor magic has yet to be applied in the solid-state power conversion realm (i.e. generating megawatts via thermionic devices).