avsm

I use Gandi for dozens of domains. This incident aside, they've been reliable and undramatic, and I don't mind paying a small premium for something as important as DNS.

Has anyone else been impacted by the sale of Gandi to Your.Online [1]? I hadn't even noticed this until the post above, but it at least looks like the acquirer is still a European company.

[1] https://your.online/press-release/

This project really should be resurrected; I'll try to find a student for this in Cambridge for the next academic cycle! I was just thinking a few days ago [1] about how a webassembly architecture for Linux brings out many of the same issues: you need to recompile apps, but then have a very flexible FFI to the outside world.

The tombl Linux-wasm [2] seems like it could be a path to upstream many of the ideas in UKML, but with a new target arch behind it so it's not just a performance boost but also a portability one. The browser demo is pretty impressive: https://linux.tombl.dev

[1] https://anil.recoil.org/notes/wasm-on-exotic-targets [2] https://github.com/tombl/linux

As the mentioned acquiree, I agree :-)

> I'm surprised the unikernel community hasn't focused on this application sooner.

There just wasn’t a compelling usecase before agents to need a serverless browser. And Chromium is basically an OS-in-a-box already, and Unikraft has now matured enough to supply the unikernel scaffolding to link it all together (coordinated by Dagger, it seems!)

Peertube has been a fantastic way to mirror videos from Youtube and have embedded versions that are ad- and cookie-free. It's got support for mirroring channels from the other video providers which makes it a breeze to run. I've deployed it for various communities:

- https://watch.ocaml.org (for all the OCaml language talks over the past 20 years)

- https://watch.eeg.cl.cam.ac.uk (for our research group on environmental science)

- https://crank.recoil.org (for my personal talks and videos)

The fediverse integration is a minor plus, but the permanent nature of the video hosting. My only wish is that there was a way to bidirectionally mirror view counts, so that the Peertube mirrors don't look like a ghost town.

There was a Commercial Uses of Functional Programming workshop talk back in 2009 where the engineers behind Facebook Chat talked about their use of Erlang.

http://cufp.org/2009/functional-programming-facebook.html

The abstract covers the pros and cons behind using Erlang back then pretty well.

Effects are included in OCaml 5, but considered experimental. This doesn't stop us from building libraries that take advantage of them internally, in order to provide really nice external interfaces.

The best developed one is "eio", which uses effects (and io_uring on Linux) internally in order to provide a really high performance, direct-style IO library for OCaml. You can walk through some of the code here: https://github.com/ocaml-multicore/eio#getting-started

Also a video talk about our experiences with using effects for writing parsers, from the OCaml Workshop last year. https://watch.ocaml.org/videos/watch/74ece0a8-380f-4e2a-bef5...

The libraries are embedded in quite a few places; for example it's used in every single Docker for Desktop instance:

- https://mirage.io/blog/2022-04-06.vpnkit

- https://www.youtube.com/watch?v=zqFDEDl5Zes

- https://github.com/moby/vpnkit

- https://speakerdeck.com/avsm/the-functional-innards-of-docke...

And then there are deployments in QubesOS, Xen, Nitrokey, etc. Bear in mind that MirageOS is a library OS, which doesn't just mean "compile to a standalone kernel instance". It just means that it can be linked into all sorts of embedded environments, some of which are the solo5 or Xen backends you see in talks.

More on the Mirage blog: https://mirage.io/blog

This all started because we wanted to _get away_ from the need to fork or run multiple processes, since that's so hard in a variety of hardware architectures (like mobile or embedded).

Other researchers also share the dislike of fork... https://www.microsoft.com/en-us/research/uploads/prod/2019/0...

A 2010 paper where I sketched out some of the early ideas around "multiscale" is here: https://anil.recoil.org/papers/2010-bcs-visions.pdf

It took a little longer than I'd planned, but thanks to the hard work of so many MirageOS contributors, now's a pretty good time to glue back personal containers and self-hosted data management infrastructure again! Unikernel-based messaging has really come together in the past couple of years: https://tarides.com/blog/2022-03-08-secure-virtual-messages-...

Just substitute 'microservice' with 'unikernel' and you do broadly the same things. There's a prometheus library that you link with the MirageOS unikernel and it exports using that: https://github.com/mirage/prometheus

No FAQ for this sort of thing yet, but we should start assembling one sometime soon. Questions like this very welcome on the discussion forums: https://discuss.ocaml.org/t/ann-mirageos-4-0/9598 to help us get started.

There's a nice collection of unikernels over at: https://github.com/roburio/unikernels and https://github.com/tarides/unikernels for various infrastructure pieces (like https, smtp, dns, ip filters, etc) that are good to crib from for your own infrastructure.

It's too early to say. Eio is still under very active development, so we'll have to see how it goes. The more useful feedback we get on it now, the more likely it'll be submitted for consideration in the OCaml stdlib when appropriate.

work has already begin on a direct-style IO library that internally uses effects. See:

- https://github.com/ocaml-multicore/eio#readme for more information on the Eio library - https://watch.ocaml.org/videos/watch/74ece0a8-380f-4e2a-bef5... is a short talk on experiences using effects with some nice motivating examples

Note that Eio is more than just a direct replacement for Lwt and Async. We couldn't resist using some of the experiences also gained from the MirageOS (mirage.io) unikernel framework in EIO. This means that the backends are highly optimised to use the best syscalls available in the OS (e.g. io_uring by default in Linux). If you write your applications to use Eio natively, then performance is very high so far. The ergonomics of programming in it also compare favourably to using monadic concurrency.

That's right -- we first wanted to establish that existing OCaml code wouldn't be adversely impacted. There are various efforts ongoing to build interested (locked and lock-free) concurrent data structures, so that will inform the parallel performance results. Nothing published yet.

(co-author of the OCaml memory model paper here)

The details of the 'LDRF' (local data race freedom) property are described in detail here: https://anil.recoil.org/papers/2018-pldi-memorymodel.pdf

The performance numbers are in the paper abstract: "our evaluation demonstrates that it is possible to balance a comprehensible memory model with a reasonable (no overhead on x86, ~0.6% on ARM) sequential performance trade-off in a mainstream programming language". It's a little higher on PowerPC but still very usable, and RISC-V overheads should be roughly comparable to ARM.

(Coauthor of RWO here)

We are just finishing edits of a few chapters (the tooling, testing and GADT ones), and then it’ll be off to the publishers early in the new year. The online one is therefore pretty up to date.

There’s also a thread on the OCaml forums on this topic with more suggestions: https://discuss.ocaml.org/t/how-do-you-stay-productive-in-oc...

You may find the “nnpchecker” configure option in 4.13.0 useful. This will print a detected use of naked pointers to stderr, which can hopefully be triggered by test suites.

Noalloc and registering roots should all be the same, as are callbacks (for sequential code, which all existing code will be)

Yes that should be fine. For incremental installation, there is a `dune build -x` which builds using a particular cross-compilation toolchain. MirageOS builds everything in one go, but it can be broken up too (as opam itself does).

Very handy, thanks. I notice that the openxt-ocaml-platform repo uses ocamlbuild/ocamlfind. Life should get a lot easier when we move that to use ocaml/dune directly -- cross-compilation really needs a lot of help from the build system, and dune is moving fast to support these workflows.

That's a pretty reasonable approach. We publish multiarch base images for OCaml/opam regularly (e.g. see `latest` at https://hub.docker.com/r/ocaml/opam2/tags or https://hub.docker.com/r/ocurrent/opam/tags for slimmer CI images).

OCaml can generate binaries that do not depend on Cygwin runtimes, and that's a common way to ship binaries. There are a number of development shops who have been doing that for years.

For development on Windows, Cygwin is commonly used for "casual" use, and can be tricky to get working if you're not familiar with the quirks of that toolchain. That's going to improve quite rapidly though: I just announced that our development focus from the OCaml Platform for opam (our package manager) is to get Windows support working end-to-end.

There aren't any real blockers to Windows support except for building a critical mass of developers to smooth out the thousands of details that get in the way right now: you can find more here: https://github.com/ocaml/opam/wiki/opam-2.2-slides.pdf

From a compiler developer's perspective, the number of variations of toolchains available on modern Windows is absolutely mindboggling. I'm looking forward to switching to Windows 10 from my OpenBSD desktop and getting familiar with all of this. Last time I seriously used Windows (95), the desktop eSheep app all the rage. I'm informed that it is now available for Windows 10: https://www.microsoft.com/en-us/p/esheep-64bit/9mx2v0tqt6rm

This is indeed on our development list for this winter to integrate more. Over in MirageOS (mirage.io), we've been working hard on getting RISC-V support in shape, so we can build unikernels for bare metal embedded devices with a simple "dune build @riscv-solo5".

As part of that work, a few things are going on:

- use "dune workspaces" (essentially a cross compilation context) to build all C bindings in OCaml packages with custom CFLAGS/LDFLAGS. Ongoing work here: https://github.com/mirage/mirage/pull/1153

- upstream in OCaml, the core developers have specced out the design for integrating cross-compilation more naturally into the compiler.

- OCaml 4.11.0 has just been released with a native code RISC-V backend (and of course has had ARM for years).

Mirage doesn't really use OpenEmbedded or Yocto, but I expect it should be straightforward to embed the workflow in there once it's stable upstream in Mirage itself. Any pointers to something I should read about how to get involved in Yocto?

ogit isn't really intended to be a CLI replacement for Git. As the comment above implies, it's a support library for higher-level applications or libraries that use git, such as irmin.io

Making this go fast and suitable for use in the Irmin (irmin.io) branching db has been a fun and multi-year effort.

See https://discuss.ocaml.org/t/ann-ocaml-git-2-0/2740 the discussion on how ocaml-git 2.0 came to be, and some of the libraries and abstractions that had to be developed for it. A lot of those are now being applied to other protocols in MirageOS, such as the new e-mail stack (who would have thought MIME parsing would be so difficult? https://github.com/mirage/mrmime)

Congratulations on the launch, and for your support of open source projects! I'm interested in extending our bulk builds of the OCaml language community packages (https://opam.ocaml.org) to Windows. Do Azure Pipelines support Windows Docker containers yet? Having somewhere to robustly build native Windows images is quite difficult at the moment.

One of the other things on my wishlist is for some sort of standard for all these CI/CD pipeline agents. So far, on our build cluster we have BuildKite, GitLab, Azure Pipelines, <insert CI provider> agents, each of which have slightly different platform support. It's particularly challenging when trying to extend our CI to slightly less popular operating systems such as OpenBSD or exotic or newer architectures such as ppc64le or RISC-V. It wasn't clear from the Azure documentation whether or not the agent is open source or not.

Today, we use a combination of BuildKite and GitLab to coordinate the multi-OS, multi-CPU-arch Docker images that then get pushed to the Hub. The first provider to give us a one-stop coordinated multiarch build pipeline will save open source projects like OCaml so much trouble for testing and publishing our releases!

I can assure you that the OCaml maintainers do appreciate the value of friendly error messages, despite your drive-by comments about our ethos.

However, unfinished patches cannot be merged into our mainline tree, and this PR https://github.com/ocaml/ocaml/pull/102 needs some attention to rebase it to trunk, and also to get feedback from testers who may have tried the opam switch and have comments. (associated writeup https://arxiv.org/abs/1512.01897)

Help welcome on this particular patch, and other newer ones from Reason that can be ported over to OCaml and submitted as normal feature improvements.

This is why we are working on integrating TUF (The Update Framework) signing into the OCaml OPAM package manager. See https://github.com/hannesm/conex-paper/blob/master/paper.pdf for the talk from last year. There's one more iteration required on the implementation before we're happy with it, but we are aiming to get this live on the OPAMv2 package repository some time in 2018 for all the publicly available OCaml packages.

OPAMv2 also exposes sufficient hooks during the build process for using OS sandboxing during builds, and disconnecting network access/etc. It would be nice to factor this out to be more OS independent (e.g. for all the `unshare` tricks on Linux, or the sexp-format for sandboxing on OSX) in the future.

Multicore is picking up pace, and there will be a paper on the formalised memory model at this year's OCaml Workshop in Oxford in September.

There are a series of milestones to hit: the runtime GC, the memory model, the low-level programming model using one-shot continuations, and how it affects libraries running over it (e.g. algebraic effects). Each of these have associated papers and talks (see the ocamllabs.ionews section), so it's not quite fair to compare it to Duke Nukem Forever :-)

Some quick links to recent papers: - memory model: http://kcsrk.info/papers/memory_model_ocaml17.pdf - programming model: http://kcsrk.info/papers/awkward_effects_ml17.pdf

Do you have a bug reference for this OCaml issue? It works fine here for me on OSX 10.12.

(author of the talk here)

This talk was more of a roundup of interesting uses of library hypervisors. The two are:

- the HyperKit/MirageOS use in Docker for Mac and how it can augment a native client application

- the far more general new Solo5 backend that IBM/Docker/Cambridge have contributed to MirageOS3, which shows how to use a library hypervisor to build a very Unix-like unikernel experience. MirageOS developers have also taken advantage of the libraries to add support for FreeBSD/bhyve very easily [1], so these new backends and libraries are increasing MirageOS library portability quite a bit.

[1] https://github.com/Solo5/solo5/issues/61

(I've just come back from Berlin and Docker Summit and have a ton of interesting questions and FAQs from attendees there. Will try to write up a blog post expanding on this presentation ahead of the MirageOS3 beta!)

> the only usage of "unikernel infrastructure" is essentially to hack around elevating privileges for networking in macOS

(author of the talk here)

That's correct, although I view that in a less dismissive light than your comment above suggests :-)

The "unikernel infrastructure" movement hinges on turning as many systems layers as we can into libraries, so that they can be repurposed in the future in different contexts. This can also include using them in _exactly_ the same way as you use systems today (e.g. compiling unikernels to Linux binaries pointing at a socket stack).

But when a problem comes along that requires challenging conventional layering, having unikernel libraries is a game changer. Without this, we would have to maintain a much more heavyweight emulation system, which in turn drives up complexity and resource usage. In the D4Mac/Win case, putting in the VPNKit library layer gives us a lot of flexibility in mapping to existing socket stacks, and also lets us open a broader conversation about what network bridging should look like in the next generation OS stack. Whatever that answer is, we know that solution isn't what we are forced to live with today -- root level privilege and importable kernel modules required to shift Ethernet packets around. Perhaps the answer is something more like FreeBSD's declarative Netgraph framework, or an OSX-style launchd socket registration system that works across higher level protocols including TLS, or something else entirely different. But until then, the unikernel infrastructure lets us solve the immediate systems problem at hand without letting that design decision vomit all over the rest of the application stack and burden us with a ton of technical debt.

> It sounds like MirageOS was only used because it was a reasonably portable network stack

It also happened to be the one that many of the team had authored, and written in a high-level enough style that we knew how to invert it. Nothing dramatic here, just picking the shortest solution to get shipping software that solved real user problems (in this case, VPN and firewall support on OSX/Win).

I expect that pulling off the same trick using several other user-level network stacks is also not hugely difficult, but I haven't used lwIP for some years to find out. The Rump Kernel NetBSD stack is probably the best alternative to start with, or the HalVM TCP/IP stack. The IncludeOS C++ one is also rapidly becoming a mature candidate.

Use mprotect(2) on the region of memory that the program wants to make executable. http://man.openbsd.org/OpenBSD-current/man2/mprotect.2

This is good portable programming practise anyway...