There are C compilers out there that work like this (like, say, the Plan 9 C compiler) and they rarely ever get used in practice, because GCC and clang -O0 are fast enough. I think making a second compiler backend just to eliminate the IR generation step isn't going to be the solution we want in the long run.