avsm

OCaml (the language I use in the post) is an extremely memory safe language, but there are plenty of higher level logic issues that spring up.

LLMs are extremely good at finding corner-case vulnerabilities in C bindings even within a memory safe language; see for example the fixes in an OCaml crypto library here: https://discuss.ocaml.org/t/the-series-of-mirage-crypto-rele...

Custom infrastructure; I use my site as a playground for various things over the years. It ran as the first MirageOS unikernel for many years, and recently it's been running on a zero-allocation OxCaml webserver https://anil.recoil.org/notes/oxcaml-httpz to help me get to grips with the new extensions.

There's a full terminal implementation of Bonsai as well. I actually use it in my personal workflow these days to manage my contacts database! https://anil.recoil.org/notes/aoah-2025-9

Fun is, by far, the biggest reason I can identify why I work on some open source projects and not others these days! I think it was also true in the past, but to a lesser extent. Docker circa 2015 never felt like a job at all and the early Dockercons werre a thing of legendary entertainment!

A couple of things have possibly conspired to make more established projects less fun in the past few years:

- we're getting older; as maintainers age, I find it's just harder to make time to go to hackathons and socialise with the younger generation, much as I would love to. Too much real life stuff on. The pandemic also established a 'before/after' and conferences just haven't been the same level since for me. I'm not sure if I changed or the conferences did.

- obviously LLMs, since they add a layer of interferences between both the demands on my maintainer time (LLM generated PRs) but also how much I need to interact with other people to solve a problem. I made plenty of online friends over the past three decades figuring stuff out together over some obscure library, but group debugging seems less important now if you can chuck it at Fable.

yep, I host two separate Tangled knots; one for my personal use and another for work at the Cambridge Computer Science department. Having large git repos on a server near me is great, and because I can sync the bare git repos it’s easy to run a local forge as well.

I looked at this last year when it came out in Tahoe: https://anil.recoil.org/notes/apple-containerisation and this looks like more of the same.

They've now added a WSL-style virtual machine layer, but there's no x86 container story (Apple's killing Rosetta) so I imagine some qemu shimming will be required.

There's still no equivalent to VPNKit or GVisor for networking so you'll be bridging I think. See: https://cacm.acm.org/research/a-decade-of-docker-containers/ for how Docker for Mac does this

I can't spot any support for dynamic memory ballooning to prevent the hypervisor from gobbling up too much memory. We've had this in Xen since forever! https://xenproject.org/blog/ballooning-rebooting-and-the-fea...

And, most obviously: NO SUPPORT FOR MACOS. This is the single feature that only Apple can do, and they're choosing not to implement it deliberately, and it's so stupid given the pains we all have to go through to implement CI for macOS. In the land of OCaml, we were forced to implement a custom ZFS snapshotter to get reasonably cost effective macOS CI for our package repository: https://tarides.com/blog/2023-08-02-obuilder-on-macos/. This was fun to build, but it sucks to have to maintain it.

Also, I'm really curious what the GPU passthrough story here is for LLMs, since the Apple Silicon -> Linux kernel support is gated on Asahi's support, but that's been lagging beyond M2 due to the efforts of reverse engineering.

Do better for your developers, Apple. This is a half-baked sweep across third-party software without addressing the core needs around your own operating system.

Nice work! Did you ever open any open source any of it? Looking at your OCaml wishlist from back in 2017, some stuff has improved and some is on its way:

- Support for read-only BigArrays (or sections) : we're starting to switch to just using bytes/string in OCaml 5+ now, since the larger allocations go into malloc'ed pools and do not relocate, so they can be used as part of an FFI (without the Bigarray C value overhead)

- More support for floating-point numbers (exceptions, representation exploration): OxCaml has some of this now! https://oxcaml.org/documentation/miscellaneous-extensions/sm...

- Syntax for extended BigArray indexing: now supported in OCaml https://ocaml.org/manual/5.4/indexops.html#ss:multiindexing

- LaCaml remains too low-level (non-functional) and unreadable: still remains the case, but OxCaml's got initial support for SIMD https://oxcaml.org/documentation/simd/intro

- BigArray and floating-point I/O remains difficult (we would like: I/O to channels, efficient representation retrieval): much easier now with OCaml effects to build custom fast serialisers (see https://github.com/ocaml-multicore/eio)

- Native top-level: ocamlnat is (I think) shipped in OxCaml, but you can also run a wasm toplevel

I must confess to also using agents to do most of my OxCaml annotations: https://github.com/avsm/ocaml-claude-marketplace/tree/main/p...

There's not that much downside since the annotations only change the performance characteristics of the program, and the static type system rejects inconsistent annotations.

Do you have a link to your talk? I'm also curious if you did any GHG measurements, or it was part of the control stack. We wrote the XenServer stack in OCaml back in 2004, and that made it into orbit in 2017 (I think it did, anyway: https://www.theregister.com/offbeat/2017/05/12/space-upstart...)

Yeah; all the really dangerous extensions are gated behind flags. But there's still a very significant number of optimisations available by default that just work well. I've taken to compiling my normal OCaml code with OxCaml these days to get a free speed boost (but buyer beware: the dependency management can be tricky; I have a giant monorepo to help out https://github.com/avsm/oxmono)

The TL;DR here (https://ccsds.org/Pubs/350x9g2.pdf) seems to be "AES GCM", but with lots of lots of legacy protocols due to older birds in the sky. DTLS or HTTP3 would seem to be a better choice these days...

The big win here is having a GC by default, with the ability to reduce heap allocations (via stack) just by adding in more typing annotations.

    Switching to OxCaml with exclave_ stack_ annotations drops 
    p99.9 latency from 29 ns to 9 ns per packet on the dispatch
    hot path, and removes GC pressure entirely (394 minor GCs to
    zero over 25 million packets). Throughput is comparable [...]

I got a similar result with my 'httpz' stack a few months ago (https://anil.recoil.org/notes/oxcaml-httpz) which my website's been running on without drama. And, I gotta say, OxCaml's a surprisingly robust compiler for being packed full of bleeding edge extensions: not a single crash on my infra is attributable to a compiler bug (plenty of bad OCaml code, but not due to a compilation bug)

The elephant in the room here is that there are hundreds of millions of embedded devices that cannot be upgraded easily and will be running vulnerable binaries essentially forever. This was a problem before of course, but the ease of chaining vulnerabilities takes the issue to a new level.

The only practical defense is for these frontier models to generate _beneficial_ attacks to innoculate older binaries by remote exploits. I dubbed these 'antibotty' networks in a speculative paper last year, but never thought things would move this fast! https://anil.recoil.org/papers/2025-internet-ecology.pdf

> If the write ups are any useful, it generally appears here or reddit and I often link back those discussions in the articles

Totally agree, I do the same as well on my site; e.g.: https://anil.recoil.org/notes/tessera-zarr-v3-layout

There are quite a few useful linkbacks:

- The social urls (bluesky, mastodon, twitter, linkedin, hn, lobsters etc) are just in my Yaml frontmatter as a key

- Then there's standard.site which is an ATProto registration that gets an article into that ecosystem https://standard-search.octet-stream.net

- And for longer articles I get a DOI from https://rogue-scholar.org (the above URL is also https://doi.org/10.59350/tk0er-ycs46) which gets it a bit more metadata.

On my TODO list is aggregating all the above into one static comment thread that I can render. Not sure it's worth the trouble beyond linking to each network as I'm currently doing, since there's rarely any cross-network conversations anyway.

From another comment below, it's just a nice short title to convey that we're going back in time and not one to set your watch by.

    We first submitted the article to the CACM a while ago.
    The review process takes some time and "Twelve years of
    Docker containers" didn't have quite the same vibe.

(The CACM reviewers helped improve our article quite a bit. The time spent there was worth it!)

It's not as easy; a block device has to be bootable and so usually bundles a kernel (large). And because the filesystem inside is opaque, you can't do layering like Docker does easily via overlayfs and friends. libguestfs does a heroic job of making VM images easier to manipulate from code, but it's an uphill battle...

cool! What services have you shipped as unikernels? Docker doesn't have to be an alternative; it can help with the build/run pipeline for them too: https://www.youtube.com/watch?v=CkfXHBb-M4A (Dockercon 2015!)

> but omission from the article stands out.

(article author here)

Apple containers are basically the same as how Docker for Mac works; I wrote about it here: https://anil.recoil.org/notes/apple-containerisation

Unfortunately Apple managed to omit the feature we all want that only they can implement: namespaces for native macOS!

Instead we got yet another embedded-Linux-VM which (imo) didn't really add much to the container ecosystem except a bunch of nice Swift libraries (such as the ext2 parsing library, which is very handy).

> I don't think SLIRP was originally for palm pilots, given it was released two years before.

That's a mistake indeed; "popularised by" might have been better. Before my beloved Palmpilot arrived one Christmas, I was only using SLIRP to ninja in Netscape and MUD sessions onto a dialup connection which wasn't a very mainstream use.

And those lightweight VM base images are possible because Docker applied a downward pressure on OS base image sizes! Alpine Linux doesn't get enough credit for this; in addition to being a great base image, it was also the first distro to prioritise fast and small image creation (Gentoo and Arch were small, but not fast to create).

Those are global to the machine; generally not an issue and seccomp rules can filter out undesirable syscalls to other containers. But GPU kernel/userspace driver matching has been a huge headache; see https://cacm.acm.org/research/a-decade-of-docker-containers/... in the article for how the CDI is (sort of) helping standardise this.

Thanks for the kind words! I've been prodding @justincormack to resurrect the single most fun OS unconference I've ever attended -- New Directions in Operating Systems (last held back in 2014). https://operatingsystems.io

Some of those talks strangely make more sense today (e.g. Rump Kernels or unikernels + coding agents seems like a really good combination, as the agent could search all the way through the kernel layers as well).

We've given up on native Windows containers in OCaml after trying to use them for our CI builds for many years. See https://www.tunbury.org/2026/02/19/obuilder-hcs/ for our recent switch to HCS instead. Compared to Linux containers, they're very much a second-class citizen in the Microsoft worldview of Docker.

Docker broke out the build layer into a separate component called BuildKit (see HN discussion recently https://news.ycombinator.com/item?id=47166264).

However, Dockerfiles are so popular because they run shell commands and permit 'socially' extending someone else shell commands; tacking commands onto the end of someone else's shell script is a natural process. /bin/sh is unreasonably effective at doing anything you need to a filesystem, and if the shell exposes a feature, it has probably been used in a Dockerfile somewhere.

Every other solution, especially declarative ones, tend to come up short when _layering_ images quickly and easily. However, I agree they're good if you control the entire declarative spec.

An extremely random fact I noticed when writing the companion article [1] to this (an OCaml experience report):

    "Docker, Guix and NixOS (stable) all had their first releases
    during 2013, making that a bumper year for packaging aficionados."

Now we get coding agent updates every week, but has there been a similar year since 2013 where multiple great projects all came out at the same time?

[1]: https://anil.recoil.org/papers/2025-docker-icfp.pdf

VPNKit (the SLIRP component) has been remarkably bug free over the years, and hasn't been much of a burden overall.

There was another component that we didn't have room to cover in the article that has been very stable (for filesystem sharing between the container and the host) that has been endlessly criticised for being slow, but has never corrupted anyone's data! It's interesting that many users preferred potential-dataloss-but-speed using asynchronous IO, but only on desktop environments. I think Docker did the right thing by erring on the side of safety by default.

We first submitted the article to the CACM a while ago. The review process takes some time and "Twelve years of Docker containers" didn't have quite the same vibe.

(coauthor of the article here)

Well, before Docker I used to work on Xen and that possible future of massive block devices assembled using Vagrant and Packer has thankfully been avoided...

One thing that's hard to capture in the article -- but that permeated the early Dockercons -- is the (positive) disruption Docker had in how IT shops were run. Before that going to production was a giant effort, and 'shipping your filesystem' quickly was such a change in how people approached their work. We had so many people come up to us grateful that they could suddenly build services more quickly and get them into the hands of users without having to seek permission slips signed in triplicate.

We're seeing the another seismic cultural shift now with coding agents, but I think Docker had a similar impact back then, and it was a really fun community spirit. Less so today with the giant hyperscalars all dominating, sadly, but I'll keep my fond memories :-)

I just use the OCaml Platform VSCode extension: (https://marketplace.visualstudio.com/items?itemName=ocamllab...) or the OCaml LSP server: https://github.com/ocaml/ocaml-lsp in other editors and don't really need anything domain specific.

The runtime can also use mount namespaces to support concurrent installations. Or, if there is a compilation step, the linker can not expose symbols for clashing libraries and just resolve them within the dependency chain.

The package calculus allows all of these to specified cleanly in a single form.