That may be true for peripherals, but all a compiler has to care about is the core ISA. The board zoo is very much not relevant.
People want - to borrow from the BSD world - FreeBSD levels of support for specific chipsets and features, with OpenBSD levels of support for security, and NetBSD levels of portability. These are not compatible outcomes, and folks should stop pretending that they are.