Let me suggest looking into the `devtool' utility. It's a yocto utility that enables on-the-fly work that the author enjoyed with buildroot. For instance, running `devtool modify virtual/kernel' will place the configured kernel source in a workspace directory where you can grep and modify and patch to your hearts content; on a new board, I might work for weeks in this state bringing up a new board as I patch drivers, or develop patches to play out-of-tree code over the mainline kernel. When I'm happy with my changes, I add them back into my recipe and test it by disabling the temporary workspace `devtool reset virtual/kernel' and building my recipe from scratch again.
Yocto has other amenities that ease iteration on existing boards. For one, it straightforward to cross-compile my python3 extension modules in a recipe in one base layer for my product family. Later, when I'm spinning up a derivative project, I can setup a product-specific layer to override the CPP flags, configurations, or patch the source to better target my board.
The yocto learning curve may be steeper, but the benefits of proper dependency tracking and layers far outweigh the drawbacks. At this point, if I use a board from a vendor that ships a buildroot BSP, I'll take a day to port it to yocto before moving further.