Is it theoretically possible to perform the profile generation+apply steps dynamically at runtime?
Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis.
HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name.
[0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...
[1] https://lwn.net/Articles/270081/