How much memory does a desktop Linux system waste? | Hacker News Reader

How much memory does a desktop Linux system waste?

omattos.com

My RAM has a lot of duplicate data in. This seems counter to good design practices which should keep just one copy of big chunks of identical data.

That's a question of trading space for time. Sure, we could look at each page of cache RAM and work out the best way to use all of it very efficiently in terms of space, but what is the rest of your system doing while you're doing all that? Just wasting time, twiddling its thumbs, asking "Are we there yet?"

Hadn't seen this analysis done before, so thought I'd do it.

Ideas welcome for ways to convert this data into better ways of doing things!

Every time something is copied (eg. to modify it slightly), it would probably leave the old behind. Perhaps you should try modifying this to only look at used memory, not free/cache memory?

It's surprisingly hard to find if a given memory address is free or used... After all, it might have been freed in the application memory allocator, but still part of an allocation pool... Or the allocation pool has been freed, but not returned to the kernel via MADV_DONTNEED. etc.

Besides, if one is making a copy of a big (ie. greater than a page) piece of data to make modifications, one probably should have used some kind of data structure that deduplicated the parts of the data which weren't modified - that saves the unmodified parts of the data needing to even be read/written by the CPU.

I'm thinking of low-level programming. Generally, operations on strings don't modify them in place, they are copied each time, and the old memory free()'d or garbage collected, which wouldn't bother clearing the memory.

In a perfect world, there wouldn't be any substantial amount of free memory (free memory is memory that isn't giving the user any benefit), and cache memory should be unique (if it's in the cache, it shouldn't be anywhere else in RAM).

By the way, you might like to try again with KSM (same page merging) enabled.

I would get the same results. No application I was running sets the MADVISE flag for KSM to have any effect.

Typically it's only things like qemu that set it.

Reply on news.ycombinator.com