I agree this is true a lot of the time, speaking as someone who has made many Yocto messes and later regretted. But you also can't discount all of the messes made by silicon OEMs that you are forced to deal with to use their SoCs.
E.g. I recently did a project with Yocto on a Zynq US+. Look at the meta-xilinx layerset (or, God forbid, petalinux) and tell me they haven't made a complete mess of it.
The complexity and lack of upstreaming in kernel and userspace for so many of these SoCs, coupled with non standard boot process and devicetrees means you will be forced to use these terrible OEM layers. I don't think a new build system can fix this at all, it's a cultural/commitment that needs to change from the OEMs.
At the end of the day, of you are doing any complex embedded Linux work as a small team, you must use Yocto because it's what the OEM supports. I don't have time or resource to debug DFX not working because some userspace tool was built with slightly different flags than expected by the flaky ass xilinx upstream.