Actually unladen_swallow and cperl are doing this. Compile the whole runtime to a lib<rt>.bc (trivial make rule), load this bitcode file (a single call), and add the expensive optimizations, esp. the inliner and IPO in llvm. Just the type-checks need to be done in the jit. This inliner does a lot of good magic esp. for the small RT functions. >10x faster.
The compiler is expensive though (i.e. very slow), and LLVM changed it's jit API 3 times already. <=3.4 jit, then mcjit and >= 3.6 ocrjit, all 3 of them still having major quirks.
With mcjit you cannot selectively add jit code to a module. So every single body needs to be a new module. So module != package/class/namespace. So the jitcache is a bit complicated. With the latest ocrjit you got problems in finding the symbols in the bc. The C API doesn't support the name resolver at all, and it's a major quirks with C++. LLVM's C API is really behind, but at least they don't change it that often as the C++ API.
unladen_swallow has a huge JIT overhead library, which nobody really needs.
For the cperl jit I just used 4 days so far and doesn't even link yet. I used the C API, not the better C++ API. No C++ for cperl. But it's very simple. https://github.com/perl11/cperl/blob/feature/gh220-llvmjit/j...