Hacker News Reader

Comment by logician76 | Hacker News Reader

Container size influences memory usage, which is an important concern:

http://www.stackengine.com/container-memory-usage-or-size-ma...

I'd be careful about drawing conclusions from those tests. We know that the number of bits in a container does not directly influence how much RAM it consumes. Therefore, there must be something the images are doing that consumes memory which is not happening in the "smaller" images. The key would be to find the culpable process or daemon(s).

It could well be due to things like shared libraries. A larger distro will have more options enabled, causing more shared libraries to be linked into the same running processes, and thus more shared libraries to be fully loaded into memory.

A smaller distro might even statically compile most things - Alpine does. If you dynamically link shared libraries, the whole library is loaded into memory to serve the process. If you statically link, only the actually used part of the library is included in the binary.

Statically linked binaries can't share the library in memory between each other like dynamically linked binaries can, but if all your processes are running in separate containers, they won't share those libraries anyway (unless they're all in a VM and the rarely used "samepage merging" is enabled for the VM).

Finally ... simplicity has knock-on effects. Making things simpler and smaller (not easier), and reducing the number of moving parts in the implementation, makes cleaning up more stuff easier.

That's an awful lot of conjecture. I'd wager that most of what you would actually be running in a container would not have its memory usage significantly affected by the presence or absence of optional shared libs.. I'm with the parent on this; such claims warrant research.

Not really, it was an educated guess, and then a description of how binaries and libraries work on modern unix systems.

Here's a quick demo based on the trivial example of the memcached docker images I mentioned in another thread:

  vagrant@dockerdev:/host/scratch/janus-gateway$ sudo docker run --name=mc_big --detach --publish=11212:11211 --user=nobody sylvainlasnier/memcached /usr/bin/memcached -v -m 64 -c 1024
  67c0e406245d341450c5da9ef03cbf60a8752433a4ace7471e2a478db9a62e07
  vagrant@dockerdev:/host/scratch/janus-gateway$ sudo docker run --name=mc_small --detach --publish=11213:11211 --user=nobody ploxiln/memcached /bin/memcached -v -m 64 -c 1024
  11037b69acfbc0de7601831634751cd342a7bafe9a25749285bc2c2803cc1768
  vagrant@dockerdev:/host/scratch/janus-gateway$ top c -b -n1 | grep 'COMMAND\|memcached'
    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND
   5984 nobody    20   0  316960   1192    768 S   0.0  0.1   0:00.02 /usr/bin/memcached -v -m 64 -c 1024
   6091 nobody    20   0  305256    780    412 S   0.0  0.0   0:00.00 /bin/memcached -v -m 64 -c 1024

Notice the significant difference in RES (resident set size) and SHR (shared memory). Less trivial processes will have more shared libraries and bigger differences here. Multiply this kind of result times all the contained processes. It adds up.

Sorry, I was responding to your post in the context of logician's "an important concern" assertion. You and jabl are correct technically of course.

Within the context of "an important concern" though; the difference in RES and SHR between the two is about ~330kb. I suspect most people wouldn't find that significant particularly given memcached's common use cases.

That's not really how it works. Both executables and shared libraries are mapped into the virtual address space of the process, then only stuff that is actually used will be faulted (read) into physical memory. At page granularity, so yes, there is some bloat due to unused functionality, but it's not as bad as requiring the entire thing to be loaded into memory.

Are you sure that is the general case? I have no reason to believe that this is true in general.

No I am actually skeptical myself, I was hoping somebody here would explain the real cause of their findings :)

Reply on news.ycombinator.com