It would be difficult as every package/program would need a step to generate the profile data by executing and running the program like the user would.
Alexia Massalin's Synthesis[0] (pdf) operating system did JIT-like optimizations for system calls. Here's a LWN article[1] with a summary. Anyone who's interested in operating systems should read this thesis.
HP's Dynamo[2] runtime optimizer did JIT-like optimizations on PA-RISC binaries; it was released in 2000. DynamoRIO[3] is an open source descendant. Also, DEC had a similar tool for the Alpha, but I've forgotten the name.
[0] https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&d...
[1] https://lwn.net/Articles/270081/