Comment by highwaylights | Hacker News Reader

Comment by highwaylights | Hacker News Reader

Parent Full thread

This introduces a long-standing problem that is really the main issue I have with Docker/Podman/containers on anything other than Linux - you have a dedicated VM that needs memory to be set aside for it exclusively.

This is fine if you have a 32/64GB machine, but less so on an 8GB non-upgradeable laptop.

I get it - memory is relatively cheap these days - and manufacturers that are building memory-limited devices are really only doing it to fleece you on obscene upgrade fees at the time of purchase - but it would be nice if there was a more elegant solution to this on Windows and macOS.

WSL 1 had a solution to this that clearly took a lot of work to put together, wherein they'd have a Linux kernel running side-by-side as a Windows sub-process so that the memory pool was shared. Unfortunately it might have been too much work as they scrapped it entirely for WSL 2 and just used essentially the same VM route.

If anyone knows of any projects trying to work around that problem I'd love to hear about it. If Apple really wanted to bring the development community back on board, focusing on these kind of use cases would be great, sadly it seems someone over there has taken the view that scrapping butterfly keys and the touchbar is "enough".

Say what you will about Microsoft, but they've focused really hard on developer use cases for decades, and it shows.

> If anyone knows of any projects trying to work around that problem I'd love to hear about it.

Containers are namespaced processes. These processes exec against the corresponding kernel they require. There is no workaround: if you have an ELF binary calling Linux syscalls it can only run on a Linux kernel†, so to run that you need a VM††. It's not as bad as it appears thanks to memory ballooning†††.

Conversely if you want to exec a Windows binary in a container, the Windows kernel needs to provide process namespacing features (which it does). And if you want to exec a Darwin binary in a container, then the Darwin kernel needs to provide process namespacing features (which it doesn't).

† WSL1 was implementing the Linux syscall API on the Windows kernel, which proved to be much more complex than it appears to be.

†† Or colinux (https://en.wikipedia.org/wiki/Cooperative_Linux), or user-mode Linux (https://en.wikipedia.org/wiki/User-mode_Linux).

††† https://en.wikipedia.org/wiki/Memory_ballooning

I suppose running a windows binary under wine in a Linux container also belong somewhere on this excellent summary.

> running a x86 windows binary - looking for 32bit stuff under SysWOW64 - under x86_64 wine in a x86_64 Linux container AOT translated to aarch64 by Rosetta 2 for Linux that switched the ARM M1 memory model to Total Store Ordering on a aarch64 Linux kernel under Virtualisation.framework on arm64 darwin.

FTFY ;) (and thanks!)

The challenge of running OCI containers in every OS is a bit similar to the problem of running the same binary in every OS.

I think the only true solutions are (a) OS vendors develop their own native container platforms with UX similar or better than Docker (b) OS vendors agree on some common ABI standard

> WSL1 was implementing the Linux syscall API on the Windows kernel, which proved to be much more complex than it appears to be.

I've long wondered, and again now that Chat GPT is proving so adept at coding, if this translation layer could be automated. Do humans actually have to hand-code each syscall? Or are there just enough edge conditions that can't be automated?

The hard part isn't so much writing the code as it is testing the code. The WINE server is decades old, porting API calls to another kernel isn't an entirely new or novel process. Getting all the software to work right is an ongoing process though, and it requires careful deliberation over what works and what doesn't. WINE code is still being refined to this day in response to new titles and old bugfixes.

So, I won't preclude the idea of AI helping, but I think human effort is still the bottleneck for projects like this. Even if AI could write perfect code 100% of the time, testing and troubleshooting would probably still be the larger timesink.

WSL 2 now allows for memory reclaim from the VM, essentially fixing this

    autoMemoryReclaim – Makes the WSL VM shrink in memory
    as you use it by reclaiming cached memory

https://devblogs.microsoft.com/commandline/windows-subsystem...

I think someone told me that the relies on a feature of the CPU itself and MS is the only OS vendor to actually implement it

This is supposed to be possible on macOS, but it's basically completely broken (just doesn't work) due to a bug. Apple is aware of it but unfortunately hasn't said much more.

Obviously that limits the options, but I'll still be taking one last shot at using creative workarounds to tackle the memory problem in OrbStack (another containers-on-macOS product).

VM dynamic allocation of ram has been a standard feature of KVM, hyper-v and VMware for the better part of 10 years now.

Recent versions of Docker Desktop have a "Resource saver"[0] mode. It will de-allocate memory and CPU when containers aren't being used. If there's a leaf icon next to your Docker icon then it's in resource saver mode.

I imagine over time it will get smarter too. Right now it waits for no containers to be running for 30 seconds and enables resource saving mode but who knows what could happen in the future. Maybe it can internally profile and estimate load based on evaluating runtime stats of your contains and dynamically change the VM's resources on the fly and then expose a +% over provision threshold option or a way to turn off dynamic resource saver mode.

[0]: https://www.docker.com/blog/docker-desktop-4-22

One of my issues too. The tools/ecosystem are Linux driven and doing anything on Windows requires a Linux 'stub' of some sort (VM, WSL, etc). I am comfortable with both OS's but all of my coworkers are Microsoft OS tied. 0% Linux experience. Cannot really introduce a managed kub/docker/containers into the picture without ending up being 'support' - not happening! The cloud is an option but that still leaves the developer experience (local laptop) short changed.

Edit: It has been a while since I last looked at this. Looks like containerd is, perhaps, a native option

WSL1 didn't run Linux kernel anywhere.

WSL2 doesn't really use VMs in the traditional sense. msft have invested heavily in lightweight virtualisation (eg better memory management) for certain security features, which allowed WSL to perform well enough without the maintenance/support overhead of WSL1

However WSL for Docker works differently depending on the containers configuration.

When Windows containers are being used, it is just there to keep the docker daemon happy.

Reply on news.ycombinator.com