A modest proposal: let's use only static libraries (plugins aside). Let's help KSM
efficiently merge the R/X, and the R/O memory pages of the static libraries of running processes. To maintain the current performance, let's give KSM a
hint: the hint would be a string of standardized, canonical library pathname, including name and version number.
During the (static) linking of libraries, an extra metadata item would be stored in the ELF file: the canonical pathnames of all the static libraries used, along with base addresses of R/X segments (code) and R/O segments (r/o data). Upon forking a process, the information would be provided to kernel, and passed over to the KSM, to use as basis of same-page lookup and possible merging of the pages.
This effectively inverts the current mechanism. Currently when libraries are dynamically loaded, the dynloader uses actual file pathname of the shlib as the key for operation opposite to "merging" - i.e., re-uses the already loaded R/X and R/O pages, by adding proper memory mapping.
The proposed change is three-fold:
1) extend the linker to add the metadata to ELF,
2) extend in-kernel ELF interpreter to extract the info upon exec() and friends,
3a) extend the KSM with a limited mode, where it would look up & merge only the hinted memory regions, in linear fashion, right after an exec() & friends.
An alternative to 3a), to avoid fussing about with KSM:
3b) modify the VM subsystem to extend current swap/SHM so it provides an unique address range for each static lib canonical pathname. Requesting pages from this address range would map from the shared pages, if any process already loaded one such.
To handle adversarial processes on single machine, a further extension where crypto signatures are checked is possible.