The host binary will include intermediate representation of the compute code to be compiled by the device driver. This already exists, see SYCL + SPIR-V. Intel's oneAPI is one implementation of this approach.