If you want a fixed build environment, pick a toolchain and stick with it.
If you want the latest and greatest, a willingness to rebuild your code seems like a reasonable prerequisite to me.
If you need a binary blob that can withstand toolchain versioning, use a dynamically linked library.