HNHacker News
TopNewBestAskShowJobs

avsm

1,532 karma · joined September 17, 2010

anil.recoil.org hacker @ocaml faculty @cambridge computer laboratory fellow @pembroke college cambridge hacker @openbsd
submissionscomments
avsm··on Package Managers à la Carte: a formal model of dependency resolution
(one of the paper coauthors here)

While diamond dependencies are indeed one of the big complicating factors, the implementation and engineering details that remain matter a lot too. Section 4 covers the spectrum of quality-of-life features that do introduce subtleties: for example the order of resolution, peer dependencies, depops/features. These are all important for the ergonomics of package constraint expressions, irrespective of whether diamond dependencies are present or not.

The engineering details also flow from the practical implementation constraints: it makes a big difference if solving can done in linear time or if there's a noticeable pause or (worse) you need a big centralised solver. The determinism also guides the implementation of chains of trust.

avsm··on Delimited Continuations vs. Lwt for Threads
(original author of that article here)

This is a blast from the past! And indeed, I use Eio these days to do direct-style IO; you can see an example of a HTTP/1.1 parser done in that style, and there's nary a monad in sight! https://tangled.org/anil.recoil.org/ocaml-requests/blob/main...

OCaml's effects are similar to thethe delimcc hack used in the article, except that it's nicely integrated into OCaml 5 and very high performance; basically a stack switch: https://anil.recoil.org/papers/2021-pldi-retroeff.pdf

avsm··on My fast zero-allocation webserver using OxCaml
It is a cool problem space! What I'm doing is using a single buffer for body handling (since you dispatch that away and then reuse it for chunked encoding) so it never takes unbounded stack space. This might be a bit different in HTTP/3 where you can have multiple body transmissions multiplexing; I have to look into how this works (but it's UDP as well)

What we never need to do in OxCaml is to keep a giant body buffer list in the stack; with effects, we can fork the stack any time, so the request object is shared naturally. The only way to free the stack is to return from a function, but you can have a tree of these that share values earlier in the callchain.

avsm··on My fast zero-allocation webserver using OxCaml
Good point; I've decided to simply not support HTTP/1.1 pipelines, and to have a connection pooling layer for HTTP/2 instead that takes care of this.

In OxCaml, it has support for the effect system that we added in OCaml 5.0 onwards, which allows for a fiber to suspend itself and be restarted via a one-shot continuation. So it's possible to have a pipelined connection stash away a continuation for a response calculation and be woken up later on when it's ready.

All continuations have to be either discarded explicitly or resumed exactly once; this can lead to memory leaks in OCaml 5, but OxCaml has an emerging lifetime system that guarantees this is safe: https://oxcaml.org/documentation/parallelism/01-intro/ or https://gavinleroy.com/oxcaml-tutorial-icfp25/ for a taste of that. Beware though; it's cutting edge stuff and the interfaces are still emerging, but it's great fun if you don't mind some pretty hardcore ML typing ;-) When it all settles down it should be very ergonomic to use, but right now you do get some interesting type errors.

avsm··on My fast zero-allocation webserver using OxCaml
(author here) it's actually the module system of OCaml that's amazing for large-scale code, not the effects. I just find that after a certain scale, being able to manipulate module signatures independently makes refactoring of large projects a breeze.

Meanwhile, in Python, I just haven't figured out how to effectively do the same (even with uv ruff and other affordances) without writing a ton of tests. I'm sure it's possible, but OCaml's spoilt me enough that I don't want to have to learn it any more :-)

avsm··on My fast zero-allocation webserver using OxCaml
(author here) The mutation is only for performance critical code. I'm first trying to match C/Rust performance in my code, and then transform it to more idiomatic functional code (which flambda2 in OxCaml can optimise).

It's too difficult right now to directly jump to the functional version since I don't understand the flambda2 compiler well enough to predict whta optimisations will work! OxCaml is stabilising more this year so that should get easier in time.

avsm··on My fast zero-allocation webserver using OxCaml
We do now though, with OxCaml! The local stack allocation mode puts in quite a strong constraint on the shape of the allocations that are possible.

On my TODO list next is to hook up the various O(x)Caml memory profiling tools: we have statmemprof which does statistical sampling, and then the runtime events buffer, and (hopefully) stack activity in OxCaml's case from the compiler.

This provides a pretty good automation loop for a performance optimising coding agent: it can choose between heap vs local, or copy vs reference, or fixed layout (for SIMD) vs fragmentation (for multicore NUMA) depending on the tasks at hand.

Some references:

- Statmemprof in OCaml : https://tarides.com/blog/2025-03-06-feature-parity-series-st...

- "The saga of multicore OCaml" by Ron Minsky about how Jane Street viewed performance optimisation from the launch of OCaml 5.0 to where they are today with OxCaml https://www.youtube.com/watch?v=XGGSPpk1IB0

avsm··on A Social Filesystem
Microblogging is also the least interesting part of the ATProto ecosystem. I've switched all my git hosting over to https://tangled.org and am loving it, not least of which is that my git server (a 'knot' in Tangled parlance) is under my control as a PDS and has no storage limits!
avsm··on Claude Cowork runs Linux VM via Apple virtualization framework
The Claude Code devcontainer works really well, especially the firewalling script! I had do a bit of GitHub Actions spelunking to figure out how to build binary images (with my own devtools preinstalled), which I wrote up here: https://anil.recoil.org/notes/ocaml-claude-dev

With this I have a nice loop where I get Claude to analyse its own sessions via a cronjob and rewrite my devcontainer Dockerfile to have any packages that I've started using during the interactive sessions. This rebuilds via GHActions and my fresh image the next day has an updated Claude and dev environment in a sandbox.

avsm··on Claude Cowork runs Linux VM via Apple virtualization framework
Funnily enough, we shipped the Docker Desktop VM a decade ago now (experience report at https://dl.acm.org/doi/10.1145/3747525). The embedded VM in DD is much more stripped down than the one in Claude Cowork (its based on https://github.com/linuxkit/linuxkit), and its more specialised to container workloads rather than just using bubblewrap for sandboxing (system services run in their own isolated namespaces).

Given how many products seem to be using this shipping-Linux-as-a-library-VM trick these days, it's probably a good time for an open source project to step up to supply a more reusable way of assembling this layer into a proper Mac library...

avsm··on Httpz – Zero-Allocation HTTP/1.1 Parser for OxCaml
This particular iteration is unbounded, but the next step is to pass in a GADT argument to specify which headers the application wants, so only those are parsed into a heterogenous tuple.
avsm··on Httpz – Zero-Allocation HTTP/1.1 Parser for OxCaml
In a conventional GCed language, you need to minimise heap allocations to avoid putting too much pressure on the garbage collector. The OxCaml extensions allows values to be passed 'locally' (that is, on the callstack) as an alternative to heap allocation. When the function returns, the values are automatically deallocated (and the type system guarantees safety).

This means that I can pass in a buffer, parse it, do my business logic, and then return, without ever allocating anything into the global heap. However, if I do need to allocate into it (for example, a complex structure), then it's still available.

It's kind of Rust in reverse: OxCaml has a GC by default, but you can write very high-performance code that effectively never uses a GC. There's also emerging support for data-race-free parallelisation as well.

The webserver I'm putting together also uses io_uring, which provides zero-copy buffers from kernel to userspace. The support for one-shot effect handlers in OCaml allows me to directly resume a blocked fiber straight from the io_uring loop, and then this httpz parser operates directly on that buffer. Shared memory all the way with almost no syscalls!

avsm··on Httpz – Zero-Allocation HTTP/1.1 Parser for OxCaml
(author here) I'm just adding data-race free parallelism support to this right now to switch my website over to using it! For those familiar with OCaml syntax, the OxCaml parse function is fun:

    val parse : buffer -> len:int -> #(status * request * header list) @ local
This takes in a buffer and returns an unboxed tuple on the stack, so there's no GC activity involved beyond stack management for each HTTP request.

https://github.com/avsm/httpz/blob/main/lib/httpz.mli#L154

avsm··on Fly's Sprites.dev addresses dev environment sandboxes and API sandboxes together
I just use Docker devcontainers using Anthropic's own Dockerfile as a base, and it gives me a persistent sandbox that have ports opened and work in any container environment (be it remote or local), and work in any IDE that supports devcontainers...

https://anil.recoil.org/notes/ocaml-claude-dev

avsm··on I ported JustHTML from Python to JavaScript with Codex CLI and GPT-5.2 in hours
The html5lib conformance tests when combined with the WHATWG specs are even more powerful! I managed to build a typed version of this in OCaml in a few hours ( https://anil.recoil.org/notes/aoah-2025-15 ) yesterday, but I also left an agent building a pure OCaml HTML5 _validator_ last night.

This run has (just in the last hour) combined the html5lib expect tests with https://github.com/validator/validator/tree/main/tests (which are a complex mix of Java RELAX NG stylesheets and code) in order to build a low-dependency pure OCaml HTML5 validator with types and modules.

This feels like formal verification in reverse: we're starting from a scattered set of facts (the expect tests) and iterating towards more structured specifications, using functional languages like OCaml/Haskell as convenient executable pitstops while driving towards proof reconstruction in something like Lean.

avsm··on An SVG is all you need
The JS is embedded within the SVG file and not exposed outside it
avsm··on An SVG is all you need
What was the SVG that didn't work? In Jon's example in the original post, the SVG he embeds there was one he wrote in around 2005. That's a pretty impressive run for it to render 20 years on...
avsm··on Eon – An Effects-Based OCaml Nameserver
Another extremely cool feature of Eon is that it uses Capnproto as the capability-based RPC interface to handle management. There's a schema that any client can implement here https://github.com/RyanGibb/eon/blob/main/lib/cap/schema.cap... including to provision ACME TLS certificates directly via DNS negotiation instead of the usual HTTP dance.
avsm··on Can a model trained on satellite data really find brambles on the ground?
https://www.inaturalist.org/observations/316277035, needs some id help :-)
avsm··on Can a model trained on satellite data really find brambles on the ground?
Are you thinking of _new_ fresh water sources that emerge in recent years? If you have any candidate lat/lon where this might have happened, we can take a look at the 2024 and earlier embeddings to see if we can spot it.
avsm··on Can a model trained on satellite data really find brambles on the ground?
...and if you do build this @ensocode, feel free to open a PR to https://github.com/ucam-eo/geotessera and I'll incorporate it as an example in the repo.
avsm··on Can a model trained on satellite data really find brambles on the ground?
An interesting 30x30m satellite that recently launched and is giving back data last year is EnMAP https://www.enmap.org. Hooking that up to TESSERA is on our todo list as soon as we can get our mittens on the satellite data
avsm··on Can a model trained on satellite data really find brambles on the ground?
That's should be a pretty good usecase; if you do just a few labels manually of known hotsprings you should be able to find others quite quickly using the TESSERA interactive notebook. The embeddings capture the annual spectral-temporal signature, so a hotspring should be fairly distinctive vs the surroundings.

Video of the notebook in action https://crank.recoil.org/w/mDzPQ8vW7mkLjdmWsW8vpQ and the source https://github.com/ucam-eo/tessera-interactive-map

avsm··on Can a model trained on satellite data really find brambles on the ground?
Yes it's very good fun just exploring the embeddings! It's all wrapped by the geotessera Python library, so with uv and gdal installed just try this for your favourite region to get a false-colour map of the 128-dimensional embeddings:

  # for cambridge
  # https://github.com/ucam-eo/geotessera/blob/main/example/CB.geojson
  curl -OL https://raw.githubusercontent.com/ucam-eo/geotessera/refs/heads/main/example/CB.geojson
  # download the embeddings as geotiffs
  uvx geotessera download --region-file CB.geojson -o cb2
  # do a false colour PCA down to 3 dimensions from 128
  uvx geotessera visualize cb2 cb2.tif
  # project onto webmercator and visualise using leafletjs over openstreetmap
  uvx geotessera webmap cb2.tif --output cb2-map --serve
Because the embeddings are precomputed, the library just has to download the tiles from our server. More at: https://anil.recoil.org/notes/geotessera-python

Downstream classifiers are really fast to train (seconds for small regions). You can try out a notebook in VSCode to mess around with it graphically using https://github.com/ucam-eo/tessera-interactive-map

The berries were a bit sour, summer is sadly over here!

avsm··on The Academic Pipeline Stall: Why Industry Must Stand for Academia
Applications to graduate school typically surge during times of economic downturns when industry is on its back foot. So...yes?

https://www.theguardian.com/education/2009/feb/17/rise-appli... (16 years ago) and more recently https://www.bloomberg.com/news/articles/2025-04-12/grad-scho...

avsm··on Transmitting data via ultrasound without any special equipment
We demonstrated this in 2003 using the smartphones of the era (they didn’t have good filters, so you could detect ultrasound quite easily).

See https://anil.recoil.org/papers/audio-networking.pdf sec 2.1 for the 2003 paper and some ancient videos at https://anil.recoil.org/projects/ubiqinteraction if you want some Nokia nostalgia :-)

avsm··on OxCaml - a set of extensions to the OCaml programming language.
David Allsopp had an oxcaml branch compiling on Windows a few months ago, so it’s in the queue…
avsm··on OxCaml - a set of extensions to the OCaml programming language.
If anyone's trying out the new opam switch, I found it helpful to use:

env OCAMLPARAM="alert=-unsafe_multidomain,_," opam install cohttp-lwt-unix

Because alerts are promoted to errors, they break existing package installs unnecessarily. The OCAMLPARAM environment variable just forces that alert to be disabled and allows the package installation to continue.

avsm··on Gandi March 9, 2025 incident postmortem
I use Gandi for dozens of domains. This incident aside, they've been reliable and undramatic, and I don't mind paying a small premium for something as important as DNS.

Has anyone else been impacted by the sale of Gandi to Your.Online [1]? I hadn't even noticed this until the post above, but it at least looks like the acquirer is still a European company.

[1] https://your.online/press-release/

avsm··on Unikernel Linux (UKL) (2023)
This project really should be resurrected; I'll try to find a student for this in Cambridge for the next academic cycle! I was just thinking a few days ago [1] about how a webassembly architecture for Linux brings out many of the same issues: you need to recompile apps, but then have a very flexible FFI to the outside world.

The tombl Linux-wasm [2] seems like it could be a path to upstream many of the ideas in UKML, but with a new target arch behind it so it's not just a performance boost but also a portability one. The browser demo is pretty impressive: https://linux.tombl.dev

[1] https://anil.recoil.org/notes/wasm-on-exotic-targets [2] https://github.com/tombl/linux

← PreviousPage 2 of 9Next →