Nix-Powered Development with OCaml | Hacker News Reader

Nix-Powered Development with OCaml

dimitrije.website

115 pointsdimitrijer104 comments

After a lot of hesitation and energized by a work incentive, I tried Nix last week.

I've followed the official docs, had to resort on Google et al. quite quickly to solve mundane problems (installing a channel is not on the default path of the docs for the new learner), and after much wandering and trial and error, I evenrially went to the discourse and asked what I should use between flakes, nix shell or develop... Every single answer was 1/extremely informative and nice 2/ crystal clear that Nix was in a confusing, in-between stage in its life cycle.

I very much appreciate the community and efforts but I work for a company that cannot afford the commitment that Nix requires right now. I will happily come back in 2 years and reevaluate.

I am working on the Nix project; resolving this "in-between stage" is our highest priority at the moment. I'd like to better understand two independent things:

1. What were the drivers and motivators for you to try Nix? What did you hope to achieve?

2. Was the current experimental status of the new nix CLI and flakes the primary cause of the confusion?

Thanks for your interest, it really confirms the interest I've received from the community in general.

1. I'm leading some Platform Engineering projects at my company and one is focused on Developer Experience. We have a wide variety of configurations on the engineer's laptops and the development environments are not normalized. That causes a lot of noise and pain, as one knows. Nix is, or seems to be, an opportunity to take back control on how engineers run the applications locally.

2. I'd say it has many factors. The official documentation and guides are not yet suitable for a zero-knowledge newcomer. As an example, since channels are still recommended in the docs, the command to add the default channel is not in the "Install" page. Things go from `curl <(...) | bash` to `nix-shell`, and it fails (on a M2 Mac) unless I navigate somewhere else on the docs to find that I should add the default channel first.

Second point is about the terminology. Derivation, Flakes... are quite arcane terms. At least to the layman (me).

Third, community blogs etc are covering outdated versions, some rely on flakes, others not, it's understandle but also understandably disorienting, especially when you're after some guidance that the official docs could not provide.

Fourth, the API. `nix-shell` is not a shortcut for `nix shell`. Wait what?

---

When I'm saying I'll come back in 2 years, I mean that it is clear that the situation is changing rapidly and dust needs to settle. Maybe the direction is not clear yet, and that would be fine. My one and only advice would be to only ship a feature that is properly documented.

Reading something like "We recommend to use flakes, they're the future, but the APIs are in flux, but if you're wary and want stability, here are our stable APIs" would have saved me a ton of time.

> take back control on how engineers run the applications locally.

How to piss off engineers in one statement. Developer machines are not something for you to “have control” over.

It's not about ‶controlling the machines″, it's about running `nix-shell -p our-project` and having the same setup on my OpenSuse than Jack on his Arch, Herman on his mac, Johann on her Debian, and our prod environment on RHEL.

That's a force multiplier in dev & prod, not a hindrance.

> Developer machines are not something for you to “have control” over.

That's exactly right - you want developers to set there machines up however they want. You also want a controlled enclave inside that, that is automatically set up and works consistently to make engineers lives easier and more consistent.

One approach people have taken for this is to use docker - a container with all the tools built in. Run the build commands inside this, problem solved. Using Nix achieves something similar, but with some benefits over the docker approach.

You're right -- but it's even worse DX to have to check a ton of different files, parse the outdated english documentation, guess, ask people, and hope that deps were frozen. Especially for old, mostly abandoned projects.

I've started using direnv to switch node versions for projects, based on already-existing CI configuration files, and it's fantastic. Having a full, working, stable development environment that can be shared among the team (amd future teams doing software archaeology when something breaks!) is my holy grail. I'm really excited about the future of nix.

But you're right that at the end of the day, devs should retain control. I think having clear, machine readable instructions for how to set up the environment enables that control. Forcing them to use the blessed config is not necessary. HAVING the config lets you know what you need to set up, your own way.

The only place such strict control needs to happen is in CI/CD, which just falls out of having the dev setup done.

Maybe I phrased poorly, but you didn’t try either.

Believe it or not, but the project emerges from the devs themselves, it was not even an idea before they spoke up and we had to make it a project, to support them.

> tomberek

1. Two independent use cases.

a) First was a personal use to carry my dotfiles/home config everywhere (office mac, home mac, virtual machines). Ended up configuring a flakes based home-manager setup.

b) Second is trying to get good devshell experience for the whole team. Especially in a monorepo setting with multiple languages and tools. Right now there is tons of setup required including installing right versions of java, python, nodejs, lots of instructions to get right python version, poetry install, setup aliases etc.

2. No, it is not. It didn't take much of research to conclude flakes are the way to go. Honestly, I am confused by so many responses about confusion between flakes and non-flakes as I don't think it take much effort to realize flakes are the way to go even if it is experimental.

Though, to be fair, it does hurt that two of the recommended resources - nix.dev and nix pills - do not cover flakes.

1) what would you consider to be "wow" or magic that would go beyond a+b. What is steps c+d that would be incredible value?

2) We're working on stabilization... just asking for a bit of patience.

Disclosure, I tried it in passing about 2 years ago.

> 1. What were the drivers and motivators for you to try Nix? What did you hope to achieve?

(a) Reproducible dev environments. I worked in a company back then had a big mix of different projects: Elixir, PHP, Java, JS, C#. We wanted everyone to just `cd` into a directory and have identical tools, versions of languages / DBs, everything.

(b) Ability to do transactional-like upgrades or downgrades of various pieces of software.

(c) Isolate software that usually pokes deep in the guts of the system or the user's home directory. PHP and JS were the main offenders here, especially with PHP on Linux (and Apache2 integration) you had to modify a good amount of files in various non-home-dir places. And JS is, well, JS, NPM's global `node_modules` directory is a horror to behold as we all know.

The experience was mixed. While we had one guy who was super invested in Nix and made sure to help us all install it and set it up, the day-to-day experience was lacking and people had to find ways to "repair" their setup after a few small upgrades of software. It felt like a chore. Sorry I can't give you more details but for us back then the main goal was to reduce the chores, not invent a different kind of them.

All in all, I'd say points (a) and (b) worked somewhat okay, while (c) we definitely didn't feel it did so we just migrated to Docker for some of our workflows.

> 2. Was the current experimental status of the new nix CLI and flakes the primary cause of the confusion?

Again, my take is outdated, but as a burned out senior dev my criticism boils down to the following:

I don't want to care about "flakes" at all. I want all tools to dispense with the cutesy pun-y terminology already and realize that people use them for work.

Give me one `nix` command with subcommands and good help outputs (and good centralized docs website). Reference: the `borg` backup program, also `git`. Before Nix gets into that state I'll not recommend it anyone.

---

I am in no way attacking you or disparaging yours and others' work but to me and my team back then it felt like Nix simply replaced one complexity with another. What drew us in was the promise of less complexity and that did not materialize.

I feel the general user experience is often lost on maintainers because they live and breathe their tool and to them the workflow is second nature. That's why I wrote this reply. Hopefully it brings some perspective.

Keep up the good work.

> 1. What were the drivers and motivators for you to try Nix? What did you hope to achieve?

I wonder how many people would honestly admit "I see it mentioned in a lot of comments/headlines from articles posted on HN and I assumed if it is good enough to be mentioned, it'll probably solve a problem I didn't know I had"

I'd call that curiosity.

"I hear about it a lot, I'm on my personal time, let's see what it's all about" seems very acceptable to me.

Not every good thing needs to be seen from an analytical perspective.

I don't disagree, but where do you draw the line from "curiosity" to "I'm letting the herd think for me and anything they dictate as interesting, I am a victim and I must investigate it because they said so"

I don't draw the line.

As a developer, in my personal activities, I don't draw the line. As a developer, in my work, I work with others to collectively agree to be on one aide or the other of the line. Even when we're retrospectively wrong. Sometimes, you can't know.

As a manager, I help engineers do the above.

I'm hesitant to write this as I don't know if you'll see it but I feel like what I have to say is different to the opinions that have been voiced so far. I will only answer for point number one.

I personally feel that Nix documentation and community blog posts sits on a gradient of theological meanderings about functional purity to actual praxis.

For one, I'm a systems engineer. What drew me to Nix was that I wanted a next-gen Ansible. A piece of software that can be used to reproduce a hosts OS state in an immutable way. For now, my homelab and laptop is all NixOS driven. I use deploy-rs and flakes to configure hosts. I can code but am not really a software engineer by trade. This is where I'm coming from.

Within the context of my established skills, Nix's documentation is awful. There were/are actions I want to accomplish that sit outside of the paradigm of Nix and when I try to accomplish those actions I feel lost.

For example, my main homelab machine has a Nomad server installed. The default Nomad package from Nix packages just installs the binary. So I needed to handle creating its data directory, its configuration, and laying down a systemd unit file. Where I got stuck was creating the data directory. I could accomplish the systemd configuration and the configuration file easily enough but I don't know how to create a directory outside of a derivation. I also don't know if a derivation is even needed.

But I also don't really know how to fit a derivation in my current flake setup so that it's processed when I change host state. I ended up solving the problem by having systemd manage a StateDirectory and creating the unit file/Nomad conf in configuration.nix for the machine. I also don't known if a derivation would even be what's needed. Do I use an overlay to extend the Nomad package? Can I ad-hoc create directories and then shove that into a module?

My point being is that system administrators, network engineers, release engineers, etc are all different types of engineers that would be interested in Nix... but they are NOT functional programming enthusiasts. I think Nix gets attention from a lot of people who are comfortable with Linux, package management, build processes, and configuration management systems like Ansible. But there is no clear runway for those types of engineers. Hell, NixOps seems to be an abandoned repo that has no documentation whatsoever for it's 2.0 release and there are dozen different projects for remote deployment to hosts.

If I took a sysadmin and told them "please codify OS state using Ansible" then they're likely to come back with a solution that could satisfy a technical need within an organization. If I took that same sysadmin and told them to use Nix to solve the same problem then they're likely to get lost for weeks.

I'm reading these.

It seems this is where a better description of what NixOS (and it's module system) is. My suspicion is that by starting with NixOS it is harder to be introduced to the simpler concepts. It is best to think of the NixOS modules as a domain-specific-language on top of Nixpkgs which is itself a large set of conventions on top of the fairly simple Nix language. I think people should be introduced from the bottom up.

> having systemd manage a StateDirectory and creating the unit file/Nomad conf in configuration.nix for the machine

Seems like you did get what you needed working; my best advice to understand that the key abstraction being created is the activation script. Perhaps we never clearly describe all of its moving parts and how it works?

> ... but they are NOT functional programming enthusiasts

I believe the key is for the Nix community messaging/marketing to focus less on the theoretical elegance and more on the practical benefits.

> Nix was in a confusing, in-between stage in its life cycle

That's what everyone says about software that is just too complex.

I tried Nix for the first time yesterday and had a similar experience - it was just too buggy, too brittle, and too difficult. It's absolutely not worth it. The software was built to solve a difficult problem, and maybe it does that, but it became a difficult problem in itself.

For example, if you Ctrl-C while its downloading an update, you get in a weird state where nothing works. Ok, so just uninstall and reinstall right? Well, there's no uninstall. You have to manually delete everything and remove the /nix mount. Its 2023, why am I having to hunt all this down? Sure, if you reinstall it it tries to do that, but it fails every time, or it just misses things. I just don't have time for this. Nix is not even close to ready for production.

> For example, if you Ctrl-C while its downloading an update, you get in a weird state where nothing works. …

There is a new Nix installer from Determinate Systems, a company heavily invested in Nix, which should make installing Nix more atomic, reversible, and thus easier:

The Determinate Nix Installer https://news.ycombinator.com/item?id=34957953

And after getting pass the installation, Nix is great for production.

The parent talks about an interrupted update process.

Interrupted updates don't do that. In single-user mode you could maybe ctrl-c it during the final activation phase, but that's a small fraction of a second -- it's just replacing a few symlinks.

In the default multi-user mode, you can't even do that.

Yes. This is like a major benefit of nix(os), updates and everything is basically atomic. If you interrupt an update you will just have a few more downloaded folders in your nix store folder, which are trivially GC-d on the next run. This is unlike pretty much anything else.

I'll take a look, thank you.

> For example, if you Ctrl-C while its downloading an update, you get in a weird state where nothing works

Heh? Oh you mean like during installing it for the very first time? Like, what do you expect from a bash script? Is bash not ready for production? That’s quite a strong statement when you didn’t even manage to install the thing… like, at least give it an honest try before you reach such a conclusion.

To be fair, I don't think it's such a huge sacrifice from the maintainers to put a check for leftover artifacts at the start of the installation script and nuke those from orbit before proceeding.

It can be put behind a confirmation Y/N terminal line as well.

But yeah, usually bash install scripts are almost never idempotent. I feel the maintainers of those scripts just give up and IMO they really shouldn't.

>Is bash not ready for production?

Almost all bash scripts aren't

(Actually I agree with you, but I used it more as a rhetoric :D)

I don't know if this is what the GP experienced, but you can definitely get some inconsistency if you ctrl-C out of Nix while it's in single-user/daemonless mode. My sense is that single-user is really only meant as a toy and for use in unprivileged containers— in basically every other situation you want to be talking to the nix-daemon, which maintains proper state and won't get interrupted.

> if you Ctrl-C while its downloading an update, you get in a weird state where nothing works

Something else is wrong here. I've aborted updates hundreds of times without problems. Uodates are atomic and this shouldn't happen.

My problem space (building ROS packages for robots) is difficult enough that it was worth it getting over that hump for me, but I totally understand this perspective.

I'm hopeful that in two years, the community will be much more aligned around flakes, and the docs (at least the onboarding parts) will have a much clearer "happy path" oriented around accepted workflows like direnv and home-manager.

> That's what everyone says about software that is just too complex.

I'm not sure you're saying this to point fingers at the software or at the people, that is very confusing :D

That being said, Nix being in-between has been discussed in the Discourse of the project here about t supposedly simple: https://discourse.nixos.org/t/nix-shell-nix-shell-and-nix-de...

It was to me very interesting to see how insiders could be diverging and potentially not being fully aware of it.

Nix is simple to the people who made it because they won't get confused and make mistakes. If you make a mistake, you can put it in a completely broken state. That is not a simple piece of software. Its overengineered and brittle.

Homebrew is an example of a simple piece of software. It just works.

> Homebrew is an example of a simple piece of software. It just works.

But, it doesn't. I've had many packages fail to build from homebrew. Also there's a recent quote I remember:

> At some point I got frustrated with homebrew because I felt like it was spending too much time upgrading when I installed new packages, and so I thought – maybe I’ll try the nix package manager!

from https://jvns.ca/blog/2023/02/28/some-notes-on-using-nix/

> If you make a mistake, you can put it in a completely broken state. That is not a simple piece of software. Its overengineered and brittle.

Do you have any examples?

What exactly are they in-between? What is changing and how will things be better in 2 years?

The Nix Flakes feature.

Previously, Nix installed its packages using 'channels'. You would add and update a channel, and then you could install packages from a channel. (This is roughly analogous to apt repositories). -- The CLI user experience for this channel-based approach is pretty terrible.

Nix flakes are a new feature where a project has a `flake.nix` file, which follows a standard format for declaring what packages a project provides. -- The CLI experience is much nicer, and has several other benefits over pre-flake Nix.

Flakes are still (for various reasons) labelled as "experimental". Much of the documentation has not been updated for flakes.

In 2 years, you'd hope that: the documentation improves, and there's less friction for adopting the technology.

My most personal example of the CLI being in-between state and the community agreeing to it is in this thread: https://discourse.nixos.org/t/nix-shell-nix-shell-and-nix-de...

The people you are talking to are either being too modest or are overly picky about what "ready" means. Flakes are 100% the way to go for new users and are production ready. They work well.

As someone who recently picked up Nix and isn’t using flakes the reason for that is that none of the official documentation seemed to be saying it was a good idea, and all I could find to the contrary was random blog posts, none of which actually explained what on Earth a flake is. I think I vaguely understand them now, but I’m hesitant to rip out the setup I’ve eventually got to and replace it with the new hotness just for the sake of it.

It is pretty old for being the "new hotness". It allows me to idiomatically and completely encapsulate my system declaratively in config. Nix isn't complete without this IMO.

You could do the same before with Niv. The new flake.nix format is pretty awful too (why do I need to declare every supported target triple separately?!).

In 2023, you shouldn’t be using nix-env or niv or shell.nix but rather just use Nix Flakes. It’s listed in the notes but this has long been where everyone has moved for a couple of years now (why it’s still “experimental” is a different question). If this is “Nix-powered”, why is everything done in the mutable shell instead of actually setting up the pure derivation so others can consume the build? Looking at opam-nix from Tweag might be a better route here. Also, why would anyone put a text editor in their devShell?

Question to others: as someone with Nix experience and interested in learning OCaml but hasn’t quite bit the bullet, what does dune actually offer me if I’m using Nix to manage dependencies already? It feels like adding an unnecessary layer of LISP when Nix and opam should already cover everything.

For a feature that isn't enabled by default, "just use Nix Flakes" hardly qualifies as best practice or even commonplace.

I've used Nix/NixOS for years and still don't have a use case for flakes. I only enabled it this week to restore some functionality to the nix command, which is still a work in progress.

Getting there was painful and required a deep dive into the documentation. In the end, it was as simple as adding this line to my configuration.nix:

  nix.settings.experimental-features = [ "nix-command" "flakes" ];

The flexibility of Nix is one of its main selling points, so it's impossible to say that "everyone" uses it in a similar way. Flakes are a welcome, but disruptive, enhancement. Hopefully, the experimental phase will end soon.

I've got a simple use case: reproducibly building a random derivation from someone's Git repo.

If they haven't pinned nixpkgs, I have no idea at what point it _was_ building successfully. It's just dumb luck whether my system's channel is a compatible nixpkgs.

If it's pinned, great, I can use that pin until I'm bothered to bring it back to mainline. At that point I'll override the nixpkgs input and bear the cost of updating and maintaining it.

Flakes make managing pins those easier, through a nice(r) CLI, and a standard format for them.

Most hardcore Nix users/developers I have met have been suspicious for Flakes for several years, so your point rings true.

That said, it feels like they are slowly coming to terms with it and just accepting it as default. Here are two examples of maintainers eventually accepting flake support on their repos after initial hesitation [1][2].

[1] https://github.com/tpwrules/nixos-apple-silicon/pull/47 [2] https://github.com/NixOS/mobile-nixos/pull/404

Counter anecdote: all of the hardcore Nix users/developers convinced me it was worth it to switch to Flakes now. They even helped me move my NixOS config as well.

It sounds like a python 2 vs python 3 kind of deal, which intuitively makes me think that you should just use flakes

I'm a relative newcomer too— started about two years ago with a Flake-only workflow on Nix 2.3. My sense in observing the debate over this time is that about 20% of the objections are technical in nature, while 80% are around governance— feeling that the feature was smuggled into the main repo as a prototype, despite there being existing external projects (niv) that were trying to accomplish a lot of the same things.

Where does it say you shouldn't use nix-env, niv or shell.nix in 2023? I would argue that there is extensive documentation and functionality for that already and it's not like it's getting deprecated. I've used Nix/Nixos for the past 4 years and still haven't looked into flakes - all the Nix stuff I use works great so I just can't really be bothered to spend time learning more, though I don't doubt it has some advantages.

I've been using Nix for about a decade, and not bothered with flakes yet. However, I also avoid nix-env and channels, as the parent suggests:

- Instead of channels, I use fetchGit with pinned revisions

- Instead of managing installed software via nix-env, I use buildEnv to collect everything I need in a single derivation. If I'm on NixOS I'll put that in systemPackages; or otherwise (e.g. on macOS) I'll use nix-env to only install that one package.

These have the advantage of being declarative files, which can be tracked in git; rather than imperative commands.

> Where does it say you shouldn't use nix-env, niv or shell.nix in 2023?

It's not in official documentation but the biggest reason is it's not reproducible by default and a new user could easily not pin nixpkgs, then think "wait... Nix isn't reproducible".

nix-shell and niv or otherwise pinning nixpkgs is pretty good, though there are some reproducibility advantages flakes has over nix-shell + niv such as:

- not allowing access to files not tracked in the git repo - forcing the user to lock their inputs with flake.lock whereas pinning/niv are optional

Also see: https://nixos.wiki/wiki/Flakes#Making_your_evaluations_pure

Meanwhile, much of the Nix world is using the composable and standardized system of Flakes so they can easily share their derivations. Flakes are behind a flag, but are a part of `nix` and all of these other tools are adding dependencies and complexity to your system. The Nix Flake shells also start up much quicker. It's really worth a look because it can simplify your setup and make it easier for others to read your Nix code and not run into something unexpected or nonstandard.

Why would I give up on shell.nix if flakes API isn't stable yet? I've got my own patterns around default.nix and shell.nix that provide better ergonomics than flakes.

Flakes have a lock file (flake.lock) which means you get to decide explicitly when to update the software. nixpkgs does not have precise rules on what updates are allowed, so nix-shell can give surprising results unless you make the effort to stick to one version of the whole nixpkgs explicitly. With flakes you can combine packages and versions explicitly.

> you make the effort to stick to one version of the whole nixpkgs explicitly.

the "effort" is just pointing to a particular checkout from nixpkgs repo

> With flakes you can combine packages and versions explicitly.

it's the same thing with default.nix/shell.nix, just having a separate lockfile changes nothing in the underlying `builtins.fetchGit` / `builtins.fetchTarball`

For your personal use, it sounds like it won't make sense.

In a team setting, enforcing that everyone has to follow purity best practices because nothing else is allowed is great.

> In 2023, you shouldn’t be using nix-env or niv or shell.nix but rather just use Nix Flakes

I've found quite a few articles on alternatives to nix-shell in the flakes world, but I haven't found one for this usecase (which is also my main Nix usecase).

Particularly, how do I have a flakes equivalent for nix-shell that isn't pure? It is convenient to be able to run git, or SSH or whatever from my development shell, and those things shouldn't be in my project build setup.

I'm not very familiar with Nix but Dune is a build system not a dependency manager. You would need it regardless.

Nix is a build system + dependency manager. So wouldn't that cover the use case? I know some packages use a Makefile with ocamlbuild or the like. I want less moving parts and less things to learn if I can.

While it might be possible, it's not very practical.

The original Nix thesis [0] talks about using Nix as a low-level build system in Chapter 10.1. And the author of opam-nix* has made an attempt to use Nix as a build system [1]. However, dune is a complicated bit of software that's not easy to replicate, and Nix isn't particularly well suited to the task.

[0] https://edolstra.github.io/pubs/phd-thesis.pdf

[1] https://gitlab.com/balsoft/tumbleweed

*Which is essentially functionality to use Nix as a dependency manger with opam-repository.

I see nix as a partial build system. It’s a more evolved makefile that will call bash scripts or other build systems to really build things in different languages. For OCaml you’ll need dune to build your projects, and that’s what nix will end up calling after setting the dependencies for you.

How would you provide incremental per-module compilation for your Nix derivation of the app?

I don't know; that's why I'm asking. Maybe buildOpamPackage already does it? https://github.com/tweag/opam-nix#buildOpamProject There's a buildDunePackage in the same region of the docs that runs `dune build` but that to me implies that dune is not needed(?). I haven't read too much into the project yet. The example (https://github.com/tweag/opam-nix/blob/4bb1a4c372f639f44fc67...) would indicate that everything is configured from just the ${PROJECT}.opam file, but I don't have enough info on the toolchain to know if it's incremental per module or what... or if ocamlfind more complicated to work with than dune.

It does run `dune build`, but the unit of persistence within Nix is defined as `buildDunePackage`, i.e. the Nix store will only know about the package rather than individual modules and a change in a single module will cause a cascading rebuild of the entire package to update the nix store.

There are tools out there for other languages that _can_ handle incremental & per-modules builds, such as purs-nix for PureScript, so if opam-nix doesn't now, I'm sure it will in the future as a lot of projects are asking for this feature.

> what does dune actually offer me if I’m using Nix to manage dependencies already?

purs-nix is a good example that even if per-module compilation is made into derivations, the rest of the remaining heavy-lifting of constraints resolution is done by the external build tool. You can think of Nix as a dependency manager, but only with a huge caveat that it doesn't actually manage constraints that come with those dependencies, especially transitive ones.

That's a good read but I'm still unclear on dune vs. ocamlfind/ocamlopt. Is dune not using those under the covers like Nix could?

While you can _build_ OCaml projects without Dune (although I wouldn't want to do that), _editing_ OCaml code I wouldn't recommend, as the OCaml-LSP depends on dune. If you're an Emacs user: Merlin AFAIK doesn't need dune.

I see. I wasn't aware that Nix was a build system and probably should have investigated a bit more before responding. Truthfully I don't use OCaml professionally but just in a hobby capacity. I believe you could roll your own using Make but the compiler invocations (which you can get dune to spit out if you want) become more complex over time the more features you add like PPXs, or `ctypes`, and unit testing, the more time you will save by using dune. tl;dr running ocamlopt directly works but will become cumbersome the bigger the project becomes

That’s still great feedback though—especially when you mention PPX support which is pretty common.

Nix can replace opam, which is the dependency manager supporting isolated ocaml environments. I generally try to use Ocaml now without opam, and just using Nix.

You'll still want dune, though I could see an argument for generating the dune files from nix expressions.

Isn't opam read upstream for the public package registry? It seems like opam is the one you need.

Dune doesn’t handle dependencies for you, you just use it to build your ocaml projects with a list of dependencies (often without specifying dependency versions, which is why dune doesn’t really work without nix)

Do you know about nix-shell --pure?

It's a good improvement, but still not as pure: https://nixos.wiki/wiki/Flakes#Making_your_evaluations_pure

If you decide to try https://devenv.sh, it has OCaml support built-in:

  $ cat devenv.nix
  { config, ...}: 
  let
    ocamlPackages = config.languages.ocaml.packages;
  in { 
    languages.ocaml.enable = true;

    packages = [ ocamlPackages.janeStreet.async ];
  }

  $ devenv shell
  ...

Recently also shipped 0.6 with container generation support: https://devenv.sh/containers/

Let me know if you give it a try :)

I was hoping this post would mention using OCaml with flambda[1] enabled. At least for my work, flambda seems to yield a ~10% speed up when compiling things. Can you get OCaml binaries with flambda enabled through Nix? With opam, I currently have to compile OCaml myself to enable it, as I am not aware of any binaries being distributed.

[1] https://v2.ocaml.org/manual/flambda.html

The nix option for it was added in 2017:

https://github.com/NixOS/nixpkgs/pull/32946

If you find it important, I'd encourage opening a PR to ask for flambda to be enabled by default at https://github.com/NixOS/nixpkgs/issues/new/choose.

Especially if the ocaml community typically enables flambda by default these days.

Seems you would need to first override ocaml and then override the ocaml used in mkOcamlPackages. [1]

  let o = ocaml.override { flambdaSupport = true; }; in
  let oP = ocaml-ng.mkOcamlPackages o (self: super: {}); in
  oP.apron

[1] https://discourse.nixos.org/t/install-ocaml-flambda-compiler...

Okay, thanks! But doesn't this still mean you have to compile ocaml (with flambda) yourself?

If the derivation has already been written (as GP implies), then compiling something yourself with Nix is essentially just a longer installation process. No downloading tarballs, no keeping track of things in /usr/local, no stow or manually adding things to your path. Nix is essentially a source distribution that pulls common binaries from caches.

I actively work on an OCaml project and couldn’t understand anything about what flambda does from its readme

I updated the link to point to the docs, which are more informative. As far as I understand, flambda is an intermediate representation of OCaml which allows for a number of optimisations and better inlining. You can check if you already have flambda enabled by runnning "ocamlopt -config | grep flambda".

If you're using opam then you can test flambda out by creating a new switch

  opam switch create ocaml-flambda ocaml-variants.4.14.1+options ocaml-option-flambda

and then running "opam switch set ocaml-flambda". (You can replace 4.14.1 with your preferred version above.)

It's a bunch of optimisation passes, I believe mostly around inlining. Ocaml is historically quite passive on that.

There is an extra superpower here that isn't mentioned - you can then trivially turn this into a CI job. This is a valid complete .gitlab-ci.yml file for the setup in this blog post:

    image: nixos/nix:latest

    before_script:
      - nix-env -f shell.nix -i -A buildInputs

    full_build:
      stage: build
      script:
        - dune build

This uses a standard docker container with Nix built in. The before_script magic then installs everything into that container exactly as it is in your development environment. Subsequent commands then run exactly as in your environment.

Of course with this approach, every CI job builds your whole environment. If this complex, and involves many compilation steps, then that is painful and you will instead want to use the nix setup to build a docker image that is used for your CI. For many projects involving small teams, that isn't necessary though and this works really well.

If you are mainly using OCaml and do not mind flakes, this was a very good starting point for me: https://github.com/brendanzab/ocaml-flake-example/blob/main/...

Using Flakes you can have a devShell, a check target and a build target neatly organized.

For those new to Nix and flakes, I created a small OCaml development shell demo based on the blog post. MyNixOS lets you create flakes using a graphical editor and generates a reproducible flake with corresponding Nix code and lockfile.

https://mynixos.com/pveierland/demo-ocaml/outputs

Run using:

  nix develop https://api.mynixos.com/pveierland/demo-ocaml/archive/latest.tar.gz

Been a while since I used nix packages or nix-shell more specifically, my issue is that the packages are "tampered" with and not as distributed, making them prone to issues. I experienced quite a few problems previously

out of curiosity, what sort of tampering/issues?

in my experience such changes are largely to accommodate nix's admittedly alien expectations (runtime dependencies living in the store, etc etc) and beyond that things behave more vanilla than almost any linux distro's repos

I really would like to use it in the enterprise. But secure boot was not easy to do the last time I used it. And i still don’t have a clue how to package things which are delivered as Deb or rpm packages (I.e. a vpn client)

And getting management approval to build a packaging team so that you are not the only person In the company to go to, is a hard sell

Well if there is no other source/binary available then you have to uncompress the deb/rpm file (both have some utility for that), look up the dependencies from somewhere, copy the extracted files over to the nix output dir, and patch-elf all the binaries to link to the nix-version of the dynamic lib.

This sounds more difficult than it is, the result will be a copied over binary file that has its “libc.so” and other dynamic libs replaced in the ELF-header with “/nix/store/hdjdewuieu737-libc/libc.so”. I recommend looking up a package in nixpkgs which has a similar install story, that’s the easiest way to write a new package.

In case you only want to run it locally https://github.com/thiagokokada/nix-alien and similar programs work fine with the binary.

Afaik it's not finished yet, but there recently has been quite a lot of activity regarding secureboot for nixos https://github.com/nix-community/lanzaboote/

Does anyone know why Nix takes up roughly 500 MB with no packages installed?

I want to try NixOS, but the “minimal” image is over twice as big as my full (terminal only) development environment using Alpine Linux. I was kind of expecting NixOS to be closer in size to a glibc-based minimal distribution (after initial install)…

Why not use the new nix develop, etc. commands?

Until the Nix team is happy to even enable flakes by default in releases, I don't think it is reasonable to expect people to use them by default.

Because nix develop is inherently a flakes-bound feature, and Nix Flakes aren't officially stable yet.

Ocaml seems somewhat hampered by this weird disconnect between upstream maintainers and Jane Street and other third party major users. Popular media for ocaml learners describes using Base, while documentation focuses on the builtins, which clearly aren’t used that often in practice.

Do I have this wrong? Why doesn’t Ocaml just merge Base (with permission) and put their best foot forward for the ecosystem?

Aside from the obvious fact that replacing the Stdlib with Base would break existing code, Base has an unstable and quite opinionated API and isn't tested on all targets OCaml supports (Jane Street is looking to drop support for 32-bit targets altogether). There would be a lot of tension to adopt it.

Meanwhile, the built-in Stdlib finally started to acquire much needed functions and modules during the late 4.x releases. My biggest worry is whether they'll keep up the pace, it doesn't even have a resizable array or, alternatively, a Clojure-like vector (but neither does Base).

So, you’d say there’s a healthy contingent of upstream devs dedicated to the health and diversity of the built in stdlib?

May I also ask, have you worked with Ocaml for any medium to large projects?

I’d be curious to hear from major users that are NOT using Base, and hear how they’re getting on. The 5.0 release especially has catalyzed my interest in Ocaml, and I’ve a few years left in me still before retirement, I’d like to pick up one last niche ‘fore I quit.

It’s complicated. From my perspective the core team was too slow to support a strong stdlib and so entreprises took over by replacing it. Now you have this ugly world where you’re split between these two things. And since everything jane street is poorly documented you are sort of forced into a poor ecosystem if you want to do serious OCaml projects.

The same thing is true with the build system dune and opam. Half-assed build system and package managers that we’re kind of stuck with.

I dream of a new cargo-like build+package manager system for OCaml that would just take over everything we currently have.

You have it wrong. The official standard library is under constant development and in practice many people use it and not the Jane Street stuff.

I don't think they're wrong

the Jane Street side are quite prolific with blog posts etc

as a newcomer to OCaml one of the first, and nicer-looking, intro resources you'll likely encounter is the Real World OCaml book https://dev.realworldocaml.org/ which unfortunately does everything using Base instead of the stdlib

Personally that didn't sit right to me and I prefer to use the stdlib by default (which seems fine and not in need of a wholesale replacement)

There definitely is indeed a problem, namely a false perception that the stdlib is a moribund relic with one foot in the grave. What I'm saying is that it's fairly healthy and gets used plenty by general OCaml users.

I totally agree

Are there some large users or projects that are committed to the stdlib and not Base that I could look into or read about?

Thanks in advance.

Reply on news.ycombinator.com