Hacker News Reader

Comment by besulzbach | Hacker News Reader

> The operating system is delivered in images that are created by utilizing the rpm-ostree project. The main benefits of the system are speed, security, atomic updates and immutability.

The article never mentions speed (or performance) again. Is the OS somehow expected to be faster because it is mounted read-only?

View on HN

No, I wouldn't expect this system to be faster in a meaningful way. Actually maybe a bit slower since fewer libraries will be cached / memory mapped between applications if they carry their own copies.

I'd love to see stats on how many libraries are really shared, I'd imagine it might be less than you hope (other than the big graphical ones)

On my laptop:

How many libraries are loaded:

    $ sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp'  |  tr -s ' ' | cut -d ' ' -f 6  |  wc -l 
    15429

How many unique library names:

    $ sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp'  |  tr -s ' ' | cut -d ' ' -f 6  |  sort | uniq | wc -l 
    872

Top 10 most shared libraries:

    sudo cat /proc/[0-9]*/maps | grep '\.so' | grep 'r-xp' | tr -s ' ' | cut -d ' ' -f 6 | awk '{count[$0]++}END{for (i in count) print i, count[i]}' | sort -k 2 -n -r | head -n 10
    /lib/x86_64-linux-gnu/libc-2.28.so 299
    /lib/x86_64-linux-gnu/ld-2.28.so 299
    /lib/x86_64-linux-gnu/libdl-2.28.so 262
    /lib/x86_64-linux-gnu/libpthread-2.28.so 237
    /lib/x86_64-linux-gnu/librt-2.28.so 227
    /lib/x86_64-linux-gnu/libuuid.so.1.3.0 205
    /lib/x86_64-linux-gnu/libz.so.1.2.11 200
    /lib/x86_64-linux-gnu/libpcre.so.3.13.3 186
    /lib/x86_64-linux-gnu/libresolv-2.28.so 170
    /lib/x86_64-linux-gnu/libgpg-error.so.0.26.1 169

EDIT: add grep r-xp to count code segment only, values in previous edit were overestimated.

When you're running a GUI those libraries also get shared. For example Qt might not be used by as many processes as libc, but it saves hundreds of megabytes of memory having that shared across my email/calendar/terminals/torrent client/windowManager/guiShell/pdfReader/webBrowser.

The GP did mention the 'big graphical ones,' but the amount of memory that gets saved was a bit stunning the first time I saw it.

That's a good idea to measure the sharing! Fortunately even if every app gets containerised, not all of those libraries will be duplicated. Specifically each app which forks on its own will still preserve sharing. For example 11 firefox processes I'm running now would share the libraries, whether it's running directly or from docker.

On linux, none of these libraries are duplicated, they are loaded once, and shared across all processes that need them.

Even when the processes are loading the libraries from different paths, in different filesystems, in different containers? How does it page in data on demand if the first container that loaded the library is killed and its filesystem unloaded?

It's not easy to share libraries across containers, unless they can be built to share a base layer in a stacked union filesystem approach.

Why can’t the linker deduplicate libraries as they’re loaded?

kernel samepage merging already exists

>Even when the processes are loading the libraries from different paths,

Well, in my original post (GGGP) I've defined "a library" as "a .so file" so what I can say is that the 872 distinct .so files used on my laptop will be shared among the different processes that use them.

If you assume the same library can be duplicated in two different .so files, then 872 is just an upper bound on the number of distinct libraries and further sharing could be done.

Eitherway, that is a significant amount of code sharing, which was the original question in this thread.

Only if they're the same version (which using the older built-in package system all packages are likely built against the same version)...

If the containerised versions of apps all have different versions of dependencies (quite likely IMO as they'll have the freedom to), there won't be any sharing.

Or even different locations. (unless they're hardlinks, or COW files probably) If it's not the same block on the disk, it's going to be duplicated in cache - whether it's the same contents or not.

If the soname is the same between the duplicate copies, and another copy is in $LD_LIBRARY_PATH it shouldn't matter about location.

Flatpak has a notion of runtimes, that are shared among applications. All the shared libraries in these runtimes will share their mmaped regions.

On top of that, ostree does deduplication based on file checksum. So if different packages ship the same binary, it will be only one copy on the disk and again, the mmaped regions will be shared among processes.

Do they mean speed of installing updates (compared to an RPM-based OS)?

Reply on news.ycombinator.com