Comment by Jhsto | Hacker News Reader

Comment by Jhsto | Hacker News Reader

Parent Full thread

> I admire how nimble you are. I aspire to write blog posts at the drop of a hat like this, but I rarely do.

Thanks! I took up blogging more often as of recent, and for me, having a manageable system is a large part of that. The last thing I want to happen on a Sunday evening is breaking some page of my website. That being said, I hope to one day make the workflow easier.

> It reminds me of a world that I've been imagining where the conclusions in scientific papers are generated as a flake outputs (and adjacent output would be the human readable thing, a PDF or whatever).

I happen to be a reviewer for software artifacts in a scientific journal, and I often use Nix here. Not that many projects do use it, but if I'm able to reproduce it with Nix, then I know the author has not missed any implicit dependencies. I like to imagine it's also useful for the authors as a feedback, whether they use Nix or not.

> I realize that you don't want to be storing mountains of data in the nix store, but it would work just as well if the output in question is an IPFS CID, to be resolved during the build instead.

I maintain separate build serves of my own using Nix integrations and the Nix cache is quite large already (so called remote builders) sitting at around 500GB. I host these at Hetzner.

I have also thought adding IPFS integration for my website, but haven't got around to it.

> I happen to be a reviewer for software artifacts in a scientific journal

That's very cool. I have a question for you.

I'm taking a bioinformatics class, despite not having the chemistry prerequisites. I'm getting a crash course in biochem, and the rest of the class benefits from having an expert in what-kind-of-quotes-to-use.

I've been thinking: would it be helpful if the care and maintenance of these compute environments wasn't left to each scientist but was instead aggregated (perhaps per-class or per-university)?

We're setting these chemists up with conda in Ubuntu in WSL in a terminal whose startup command activates the conda environment. Not exactly a recipe for reproducibility after they get a new laptop.

What if certain compute-heavy classes published flakes which the students could...

a) use while taking the class so we stop wasting time on troubleshooting ssl deps via conda

b) reference in publications after the fact. They could say:

> Here's a Jupyter notebook, download it and run it in the UCCS biochem environment like so: `nix run github:UCCS/CHEM4573?rev=16afd67`, its output lets us make the following conclusions...

I know it would be helpful for the students in the class. Do you think it would be helpful to them later on when they were publishing things?

I'm thinking about packaging the dependencies for this class, giving it to the teacher, and pitching it to the university:

> Set up a technical fellowes program. Waive tuition for us nerds and in exchange we'll support your students and faculty through the maintenance of these environments.

I don't mind paying tuition so much, but I'd like to do something to get a bit more cross pollination going between scientists in need of tech support and techies in need of something meaningful to work on.

Am I dreaming here, or would it solve some problems? Do you think I have a shot at convincing anybody?

Couple of thoughts:

> I've been thinking: would it be helpful if the care and maintenance of these compute environments wasn't left to each scientist but was instead aggregated (perhaps per-class or per-university)?

This is definitely something that Nix can abstract quite well. In my company we have [an infrastructure of computers](https://github.com/ponkila/homestaking-infra) that we manage with NixOS. We have gone over the system such that `cd` into a directory "conjures" the environment using devenv or direnv. We don't do anything too fancy yet, but we have a project commencing next month in which we start to also manage routers this way. We speculate that this will help us to do things such as follows: register new node, and it gets automatically imported by the router which establishes DNS entries and SSH keys for each user. The idea is that we could have different "views" of the infrastructure depending on the user which the router could control. For administrators, we have a separate UI created with React that pulls NixOS configuration declarations from a git repository (note: these don't have to be public) and shows how the nodes connect with each other. The UI is still under construction, but imagine this but now with more nodes: https://imgur.com/a/obBfRk0. We have this set up at https://homestakeros.com.

Depending on a project you are working on, you could then have a subset of the infrastructure be shown to the user and have things such as SSH aliases and other niceties set up on `cd` in. When you `cd` out, then your view is destroyed.

We have quite overengineered this approach -- we run the nodes from RAM. NixOS has the idea of "delete your darlings" which is having a temporary rootfs. We have gone the extra mile that we don't even store the OS on the computer, the computers boot via PXE and load the latest image from the router (though any HTTP server will do, I boot some from CloudFlare). We do this because it also forces the administrators to document changes that they do -- there is nothing worse than starting to call up people when theres is downtime and try to figure your way back up from what the mutations are. PXE booting establishes a working initial state for each node -- you just reboot the computer, and you are guaranteed to get into a working initial state. I'm personally big on this -- all my servers and even my laptop works like this. We upgrade servers by using kexec -- the NixOS configurations produce self-contained kexec scripts and ISO images for hypervisors (some stakeholders insist on running on Proxmox). I've suggested some kernel changes to NixOS which would allow boostrapping arbitrary size initial ramdisks, because otherwise you are limited to 2GB file size.

> We're setting these chemists up with conda in Ubuntu in WSL in a terminal whose startup command activates the conda environment. Not exactly setting them up for reproducibility if they ever move to a different laptop.

Python in specific is a PITA to setup with Nix, dream2nix etc., might help but it's definitely the hardest environment to set up of all languages I've tried -- even GPGPU environments are easier. Oftentimes, the only problem is not the packaging, but also the infrastructure used. For that, you could also publish the NixOS configurations and maybe distribute the kexec or ISO images.

A notable thing is that devenv also allows creation of containers from the devShell environment, which may further help your case. Researchers could reference docker images instead of insisting on everyone to use Nix.

In any case, I put some emails on my HN profile so we can also take the discussion off platform -- we are looking for test users for the holistic approach using PXE, and we are currently funded until Q3 next year.

Not sure on your issue with reproducibility using conda is. We (team of RSE working with many researchers) have had good success with storing conda environment files in git along side the code, only a few commands to get a working environment. We provide class room training to researchers and provide the training material and environments this way.

I don't have the link to the github issue handy, but I remember that the key was to ask for a lower version of python and then supply the `--force-reinstall` param. The fact that it only happened to some students was evidence that conda wasn't as hermetically sealed as nix.

My real gripe is that in that issue, the app developer couldn't really help--since it was a packaging problem--and the conda folks were unaware because the users had gone straight to the app developer. If there must be a third party doing curation, it seems to me that they should be more narrowly focused on whatever particular suite of tools enables whatever particular group of people--not on individual packages.

I know that conda lets users do this too, but I don't think that the environments compose as well as they do with nix. If you want Jim's envioronment, but with Susie's custom build of foo-tool, you can just take both as inputs, overwrite foo-tool as desired, and output composition. Your maintenance burden remains small. If conda handles environments with this kind of compositional attitude, I'm unaware of it.

> I don't have the link to the github issue handy, but I remember that the key was to ask for a lower version of python and then supply the `--force-reinstall` param. The fact that it only happened to some students was evidence that conda wasn't as hermetically sealed as nix.

Conda can be a bit fiddly, it makes life much easier if you specify the required version of python when you create a new environment.

>I know that conda lets users do this too, but I don't think that the environments compose as well as they do with nix.

Yes, though we find Conda great for most of our use cases, we still have to resort to creating containers.

Reply on news.ycombinator.com