Hacker News Reader

Comment by paxys | Hacker News Reader

It's impossible to have "native" support for Linux containers on macOS, since the technology inherently relies on Linux kernel features. So I'm guessing this is Apple rolling out their own Linux virtualization layer (same as WSL). Probably still an improvement over the current mess, but if they just support LXC and not Docker then most devs will still need to install Docker Desktop like they do today.

View on HN

Apple has had a native hypervisor for some time now. This is probably a baked in clone of something like https://mac.getutm.app/ which provides the stuff on top of the hypervisor.

Using a hypervisor means just running a Linux VM, like WSL2 does on Windows. There is nothing native about it.

Native Linux (and Docker) support would be something like WSL1, where Windows kernel implemented Linux syscalls.

Surely if Windows kernel can be taught to respond to those syscalls, XNU can be taught it even easier. But, AIUI the Windows kernel already had a concept of "personalities" from back when they were trying to integrate OS/2 so that zero-to-one for XNU could be a huge lift, not the syscalls part specifically

Exactly. So it wouldn't necessarily be easier. NT is almost a microkernel.

Yep. People consistently underestimate the great piece of technology NT is, it really was ahead of its time. And a shame what Microsoft is doing with it now.

Was it ahead? I am not sure. There was lots of research on microkernels at the time and NT was a good compromise between a mono and a microkernel. It was an engineering product of its age. A considerably good one. It is still the best popular kernel today. Not because it is the best possible with today's resouces but because nobody else cares about core OS design anymore.

I think it is the Unix side that decided to burry their heads into sand. We got Linux. It is free (of charge or licensing). It supported files, basic drivers and sockets. It got commercial support for servers. It was all Silicon Valley needed for startups. Anything else is a cost. So nobody cared. Most of the open source microkernel research slowly died after Linux. There is still some with L4 family.

Now we are overengineering our stacks to get closer to microkernel capabilities that Linux lacks using containers. I don't want to say it is ripe for disruption becuse it is hard and again nobody cares (except some network and security equipment but that's a tiny fraction).

> Was it ahead? I am not sure.

You say this, but then proceed to state that it had a very good design back then informed by research, and still is today. Doesn't that qualify? :-)

NT brought a HAL, proper multi-user ACLs, subsystems in user mode (that alone is amazing, even though they sadly never really gained momentum), preemptive multitasking. And then there's NTFS, with journaling, alternate streams, and shadow copies, and heaps more. A lot of it was very much ahead of UNIX at the time.

> nobody else cares about core OS design anymore.

Agree with you on that one.

> You say this, but then proceed to state that it had a very good design back then informed by research, and still is today. Doesn't that qualify? :-)

I meant that NT was a product that matched the state of the art OS design of its time (90s). It was the Unix world that decided to be behind in 80s forever.

NT was ahead not because it is breaking ground and bringing in new design aspects of 2020s to wider audiences but Unix world constantly decides to be hardcore conservative and backwards in OS design. They just accept that a PDP11 simulator is all you need.

It is similar to how NASA got stuck with 70s/80s design of Shuttle. There was research for newer launch systems but nobody made good engineering applications of them.

Unix 'died' with plan9/9front, which is far more advanced than Unix v7 for a PDP or a DEC, can't remember.

9front is to Unix was NT it's for VMS.

> the Windows kernel already had a concept of "personalities" from back when they were trying to integrate OS/2 so that zero-to-one for XNU could be a huge lift, not the syscalls part specifically

XNU is modular, with its BSD servers on top of Mach. I don’t see this as being a strong advantage of NT.

WSL1 didn't use the existing support for personalities in NT

XNU similarly has a concept of "flavors" and uses FreeBSD code to provide the BSD flavor. Theoretically, either Linux code or a compatibility layer could be implemented in the kernel in a similar way. The former won't happen due to licensing.

Hyper-V is a type 1 hypervisor, so Linux and Windows are both running as virtual machines but they have direct access to hardware resources.

It's possible that Apple has implemented a similar hypervisor here.

> The Containerization framework enables developers to create, download, or run Linux container images directly on Mac. It's built on an open-source framework optimized for Apple Silicon and provides secure isolation between container images

That's their phrasing, which suggests to me that it's just a virtualization system. Linux container images generally contain the kernel.

> Linux container images generally contain the kernel.

No, containers differ from VMs precisely in requiring dependency on the host kernel.

Hmm, so they do. I assumed because you pulled in a linux distro that the kernel was from that distro is used too, but I guess not. Perhaps they have done some sort of improvement where they have one linux kernel running via the hypervisor that all containers use. Still can't see them trying to emulate linux calls, but who knows.

> I assumed because you pulled in a linux distro that the kernel was from that distro is used too,

Thst's how docker works on WSL2, run it on top of a virtualised linux kernal. WSL2 is pretty tightly integrated with windows itself, stil a linux vm though. It seems kinda weird for apple to reinvent the wheel for that kind of thing for containers.

> Thst's how docker works on WSL2, run it on top of a virtualised linux kernal. WSL2 is pretty tightly integrated with windows itself, stil a linux vm though. It seems kinda weird for apple to reinvent the wheel for that kind of thing for containers.

Can't edit my posts mobile but realized that's, what's the word, not useful... But yeah, sharing the kernal between containers but otherwise makes them isolated allegedly allows them to have VMesque security without the overhead of seperate VMs for each image. There's a lot more to it, but you get the idea.

They usually do contain a kernel because package managers are too stupid to realise it’s a container, so they install it anyway.

It is as native as any Linux cloud instance.

In case you're wondering, the Hypervisor.framework C API is really neat and straightforward:

1. Creating and configuring a virtual machine:

    hv_vm_create(HV_VM_DEFAULT);

2. Allocating guest memory:

    void* memory = mmap(...);
    hv_vm_map(memory, guest_physical_address, size, HV_MEMORY_READ | HV_MEMORY_WRITE | HV_MEMORY_EXEC);

3. Creating virtual CPUs:

    hv_vcpu_create(&vcpu, HV_VCPU_DEFAULT);

4. Setting registers:

    hv_vcpu_write_register(vcpu, HV_X86_RIP, 0x1000); // Set instruction pointer
    hv_vcpu_write_register(vcpu, HV_X86_RSP, 0x8000); // Stack pointer

5. Running guest code:

    hv_vcpu_run(vcpu);

6. Handling VM exits:

    hv_vcpu_exit_reason_t reason;
    hv_vcpu_read_register(vcpu, HV_X86_EXIT_REASON, &reason);

Thanks for this ! Apple Silicon?

One of the reasons OrbStack is so great is because they implement their own hypervisor: https://orbstack.dev/

Apple’s stack gives you low-level access to ARM virtualization, and from there Apple has high-level convenience frameworks on top. OrbStack implements all of the high-level code themselves.

How does it compare to apple’s hv?

Better filesystem support (https://orbstack.dev/blog/fast-filesystem) and memory utilization (https://orbstack.dev/blog/dynamic-memory)

The screenshot in TFA pretty clearly shows docker-like workflows pulling images, showing tags and digests and running what looks to be the official Docker library version of Postgres.

Every container system is "docker-like". Some (like Podman) even have a drop-in replacement for the Docker CLI. Ultimately there are always subtle differences which make swapping between Docker <> Podman <> LXC or whatever else impossible without introducing messy bugs in your workflow, so you need to pick one and stick to it.

https://opencontainers.org/

If you've not tried it recently, I suggest give the latest version of podman another shot. I'm currently using it over docker and a lot of the compatibility problems are gone. They've put in massive efforts into compatibility including docker compose support.

Yeah, from a quick glance the options are 1:1 mapped so an

  alias docker='container'

Should work, at least for basic and common operations

What about macOS being derived from BSD? Isn’t that where containers came from: BSD jails?

I know the container ecosystem largely targets Linux just curious what people’s thoughts are on that.

Conceptually similar but different implementations. Containers uses cgroups in Linux and there is also file system and network virtualization as well. It's not impossible but it would require quite a bit of work.

BSD jails are architected wholly differently from what something like Docker provides.

Jails are first-class citizens that are baked deep into the system.

A tool like Docker relies using multiple Linux features/tools to assemble/create isolation.

Additionally, iirc, the logic for FreeBSD jails never made it into the Darwin kernel.

Someone correct me please.

> what something like Docker provides

Docker isn't providing any of the underlying functionality. BSD jails and Linux cgroups etc aren't fundamentally different things.

Jails were explicitly designed for security, cgroups were more generalized as more about resource control, and leverages namespaces, capabilities, apparmor/SELinux to accomplish what they do.

> Jails create a safe environment independent from the rest of the system. Processes created in this environment cannot access files or resources outside of it.[1]

While you can accomplish similar tasks, they are not equivalent.

Assume Linux containers are jails, and you will have security problems. And on the flip side, k8s pods share UTM,IPC, Network namespaces, yet have independent PID and FS namespaces.

Depending on your use case they may be roughly equivalent, but they are fundamentally different approaches.

[1] https://freebsdfoundation.org/freebsd-project/resources/intr...

> BSD jails are architected wholly differently from what something like Docker provides. > Jails are first-class citizens that are baked deep into the system.

Both very true statements and worth remembering when considering:

> Additionally, iirc, the logic for FreeBSD jails never made it into the Darwin kernel.

You are quite correct, as Darwin is is based on XNU[0], which itself has roots in the Mach[1] microkernel. Since XNU[0] is an entirely different OS architecture than that of FreeBSD[3], jails[4] do not exist within it.

The XNU source can be found here[2].

0 - https://en.wikipedia.org/wiki/XNU

1 - https://en.wikipedia.org/wiki/Mach_(kernel)

2 - https://github.com/apple-oss-distributions/xnu

3 - https://cgit.freebsd.org/src/

4 - https://man.freebsd.org/cgi/man.cgi?query=jail&apropos=0&sek...

Thank you for the links I will take a closer look at XNU. It’s neat to see how these projects influence each other.

> Thank you for the links I will take a closer look at XNU.

Another great resource regarding XNU and OS-X (although a bit dated now) is the book:

  Mac OS X Internals
  A Systems Approach[0]

0 - https://openlibrary.org/books/OL27440934M/Mac_OS_X_Internals

This is great! Thank you!

„Container“ is sort of synonymous with „OCI-compatible container“ these days, and OCI itself is basically a retcon standard for docker (runtime, images etc.). So from that perspective every „container system“ is necessarily „docker-like“ and that means Linux namespaces and cgroups.

With a whole generation forgetting they came first in big iron UNIX like HP-UX.

Does it really matter, tho?

Some people think it matters to properly learn history, instead of urban myths.

History is one thing, who-did-it-first is often just a way to make a point in faction debates. In the broader picture, it makes little difference IMHO.

Another reason it matters is they might have done it differently which could inspire future improvements. :)

I like to read bibliographies for that reason—to read books that inspired the author I’m reading at the time. Same goes for code and research papers!

Interesting. My experience w/ HP-UX was in the 90s, but this (Integrity Virtual Machines) was released in 2005. I might call out FreeBSD Jails (2000) or Solaris Zones (2005) as an earlier and a more significant case respectively. I appreciate the insight, though, never knew about HP-UX.

https://en.wikipedia.org/wiki/HP_Integrity_Virtual_Machines

HP-UX Vault, released with HP-UX 10.24, in 1996,

https://en.m.wikipedia.org/wiki/HP-UX

What you searched for is an evolution of it.

OS X pulls some components of FreeBSD into kernel space, but not all (and those are very old at this point). It also uses various BSD bits for userspace.

Good read from horse mouth:

https://developer.apple.com/library/archive/documentation/Da...

Thank you—I’ll give that a read. :)

Another really good read about containers, jails and zones.

https://blog.jessfraz.com/post/containers-zones-jails-vms/

WSL throughput is not enough for file intensive operations. It is much easier and straightforward to just delete windows and use Linux.

Using the Linux filesystem has almost no performance penalty under WSL2 since it is a VM. Docker Desktop automatically mounts the correct filesystem. Crossing the OS boundary for Windows files has some overhead of course but that's not the usecase WSL2 is optimized for.

With WSL2 you get the best of both worlds. A system with perfect driver and application support and a Linux-native environment. Hybrid GPUs, webcams, lap sensors etc. all work without any configuration effort. You get good battery life. You can run Autodesk or Photoshop but at the same time you can run Linux apps with almost no performance loss.

FWIW I get better battery life with ubuntu.

How? What's your laptop brand and model? I've never had better battery life with any machine using ubuntu.

Are you comparing against the default vendor image that's filled with adware or a clean Windows install with only drivers? There is a significant power use difference and the latter case has always been more power efficient for me compared to the Linux setup. Powering down Nvidia GPU has never fully worked with Linux for me.

Unless you need to have a working video or audio config as well.

If they implemented the Linux syscall interface in their kernel they absolutely could.

Aren't the syscalls a constant moving target? Didn't even Microsoft fail at keeping up with them in WSL?

Not Linux syscalls, they are a stable interface as far as the Linux kernel is concerned.

Linux is exceptional in that it has stable syscall numbers and guarantees stability. This is largely why statically linked binaries (and containers) "just work" on Linux, meanwhile Windows and Mac OS inevitably break things with an OS update.

Microsoft frequently tweaks syscall numbers, and they make it clear that developers must access functions through e.g. NTDLL. Mac OS at least has public source files used to generate syscall.h, but they do break things, and there was a recent incident where Go programs all broke after a major OS update. Now Go uses libSystem (and dynamic linking)[2].

[1] https://j00ru.vexillium.org/syscalls/nt/64/

[2] https://go.dev/doc/go1.11#runtime

arm64 macOS doesn't even allow statically linked binaries at all.

on the windows side, syscall ABI became stable since Server 2022 to run mismatched container releases

They're not really a moving target (since some distros ship ancient kernels, most components will handle lack of new syscalls gracefully), but the surface is still pretty big. A single ioctl() or write() syscall could do a billion different things and a lot of software depends on small bits of this functionality, meaning you gotta implement 99% of it to get everything working.

They didn't.

FreeBSD and NetBSD do this.

WSL doesn't have a virtualization layer, WSL1 did have but it wasn't a feasible approach so WSL2 is basically running VMs with the Hyper-V hypervisor.

Apple looks like it's skipped the failed WSL1 and gone straight for the more successful WSL2 approach.

I installed Orbstack without Docker Desktop.

WSL 1.0, given that WSL 2.0 is regular Linux VM running on HYPER-V.

I wonder if User-Mode Linux could be ported to macOS...

It would probably be slower than just running a VM.

Reply on news.ycombinator.com