Show HN: FlakeHub – Discover and publish Nix flakes
flakehub.com
flakehub.com
> FlakeHub not only indexes flakes, but provides a mechanism for fetching compatible versions based on Semantic versioning — and, unlike pinning tags in your flake.nix inputs, keeping them up to date.
> This means that you can specify that you want your input to be compatible with whatever is the latest release at the time that you add it, and get bugfixes with every nix flake update, without manually having to keep track of the releases made upstream, nor follow a development branch which may introduce breaking changes at any time.
Which is stupidly sexy to me. Semver-like is something i've been wanting so badly from Flakes/Nix. This has my attention now.
[1]: https://discourse.nixos.org/t/introducing-flakehub/32044/6
My (mostly assumed!) takeaway is that Flakes isn't perfect, and many existing users don't like that a very important part of Nix is being rushed. Conversely, many Flakes users think almost the exact opposite -- that Flakes has been taking too long and Nix is incomplete without it (i'm more in this camp).
All around though i don't have the energy to partake in this holy war, so i mostly ignore it. I'm sure i am wrong on some important details, so please don't take what i say as more than it's worth.
It is that the historical developer of nix, who was also behind flakes, moved to create a company called Determinate Systems. And this company keep pushing out products that are "ways to fix all kind of problems with nix and flakes".
Except they are done in secret then announced, while the rest of the community is trying to actually make the underlying stuff just work.
It is a bit like the guy built a half finished stuff, good enough for him, leave it in this state, then keep coming back with "solutions" that are not in nix themselves. And let everyone else clean his stuff.
Ofc the other way to look at it is that they are the only one having results and really fixing things. But I am on the side of all the other maintainers that keep having to clean up the mess.
I had no idea about this, actually. That certainly throws a wrinkle into all of it
FlakeHub brings a crates.io-like experience to Nix and flakes, with semver releases and publish-time checks to make sure the flake is easily distributable.
Check it out, let us know what you think! I'll be here answering questions all day.
You can search on the web UI, though: https://flakehub.com/search
So shouldn't flakehub be owned by The NixOS Foundation instead of being a 3rd party and proprietary? It seems to be a critical feature for flakes.
Yeah I'm not sure what would be best in this regard. Flakes are on track to becoming a critical feature for Nix, though I'm not sure Flakehub will be a critical feature for Flakes. It seems there are some issues with standardization of pinning and versioning that need to be fixed in upstream, but the Flakehub folks seem committed to transitioning to those when fixed:
https://discourse.nixos.org/t/introducing-flakehub/32044/14
As long as those standards get worked out upstream and Flakehub and similar projects follow them, and they're technically interchangeable, then they may be ok. It's if there are a bunch of different proprietary solutions with different proprietary standards where it becomes problematic.
I know that pre-flakes conventions for reusing Nix code are pretty open-ended, and naturally 'pure evaluation mode' means turning a bunch of things off. I can also see how if you are interested in integrations with stateful systems or maybe relaxing certain constraints or allowing/tracking some 'impurities', that might also seem like a step in the wrong direction.
Do you have any possibilities (or examples) in mind for how Nix might be used beyond the 'local optimum' of most current usage?
I’m sorry some folks aren’t excited about FlakeHub, but that is okay. We’ve made it because it solves real problems we’ve experienced, and I think it is going to make a big difference in the community. We’re all contributing to the project in the ways we want to, and this is an example of that.
We see flakes as the future of Nix, and we’re unabashedly working to make that true. One way we’re doing that is using flakes extensively every day and exposing ourselves to the bugs, papercuts, and pain points. We often deeply research and resolve these problems upstream, without fanfare or posting on Discourse.
I think we have the common goal of growing and improving Nix, and we’re glad to be doing it together. We would love to see a continuation of RFC-144 accepted into Nix upstream. The version resolving feature of FlakeHub is a bandaid that works until the upstream project has a comparable feature. In my opinion, it is a good thing to do experiments like this without committing the entire project to it.
Something I've noticed: `nixos/*` and `home-manager/*` are incompatible (home-manager expects 23.11 while nixos is 23.05).
You can get nixpkgs unstable like this:
nixpkgs-unstable.url = "https://flakehub.com/f/NixOS/nixpkgs/1.*.tar.gz"; I am very disappointed. Instead of focusing on actually fixing problems with Flakes and trying to stabilize them, DetSys goes full in on experimental Flakes, advertising them as the future, ignoring all problems with them and building products on top of them. Meanwhile everybody trying to help out official efforts, the Nix team, Nixpkgs maintainers, the documentation team and more are suffering because Flakes isn’t stable. It seems like DetSys has no interest in alleviating the pressure and bridging the gap that Flakes has created in the community.
And more concretely, it’s a well-known problem that Flakes suffer from an explosion of dependencies 64 with little reuse and interoperability. The design of how dependencies are locked just isn’t right. But oh surprise, that’s exactly what FlakeHub seems to help with by adding an ad-hoc third-party versioning scheme on top of Flakes. And all that centralised (wasn’t the whole point of Flakes to not be centralised?), proprietary and with seemingly no intention of upstreaming it in any way.
This all just doesn’t sit right with me :frowning:
Oh, and it has 50 likes, lol
This comment on Discourse has a lot of likes in part because people in the community recognize his username from his contributions even if they haven't worked with him.
He's not some random passerby throwing out an opinion about a project he has no stake in. He's a longtime, highly active contributor in the Nix ecosystem who cares about projects he's involved in and other efforts related to them.
You don't have to agree with his take but this is disrespectful and lame.
> people in the community recognize his username from his contributions
> He's a longtime, highly active contributor in the Nix ecosystem who cares about projects he's involved in and other efforts related to them.
> You don't have to agree with his take but this is disrespectful and lame.
Without doubling down on my viewpoint that you already think is lame and disrespectful, you don't think any of these achievements are probably mildly unhealthy/obsession level?
That said, a lot of people in the Nix community go above and beyond what could reasonably be asked of them by others, and it probably does put them at risk of problems including burnout. I'd be interested in research into emotional well-being and F/OSS involvement in general, because it could be really valuable in informing strategies for building F/OSS sustainably and even personal advice for staying balanced.
At the individual level, though, concern about this kind of thing is best left to people who don't need to look at metrics related to VCS activity to tell if someone is taking a hobby (or job) too far— trusted, close relations: family members, old friends, life partners, dear neighbors, etc.
Can we reasonably assume that somebody who pours themselves into open-source to the point of being visibly emotionally upset over the fact that somebody else made a package/service that goes in a direction you don't agree with most likely means you're sacrificing time otherwise spent in exactly what you mentioned: maintaining/having relationships with family members, old friends, life partners, etc.
How can you maintain a healthy social life + fitness life + professional career life if the bulk of your free time is spent caring so passionately about open source you're willing to argue/dedicate a ton of your free time to it?
I'm willing to bet people who operate at this level don't take care of themselves in at least one other area, whether it be non-PC-related relationships, fitness, social life, etc.
I also want to mention that I usually don't argue much about issues. Most of my time is spent on making concrete improvements to the ecosystem :)
You are right, done.
edolstra isn’t some rando either, he’s behind (if I understand correctly) the original Nix paper all the way back in 2004: https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf
I think all you're missing is my point. ;)
My point is not that it's automatically a huge deal whether Infinisil (or anyone) disapproves of this or that development in the community. But that nobody deserves to be dismissed with a backhanded I-hope-my-attempt-to-discredit-you-doesn't-discredit-you 'go touch grass' comment. And that I think within the Nix(OS) community there's basic neighborly recognition that is clearly missing here (on HN).
Totally agree that “go touch grass” is a disrespectful take.
These are a bunch of O.G. hackers who’ve invested a portion of their life into a significantly underfunded problem statement in our industry.
They deserve respect and our industry is better for them spending their time the way they did.
I’m personally very thankful these folks exist and are as passionate as they are. It gives me hope that one day I’ll work with a package manager in production that does more than just YOLO unpack an archive file into a global namespace followed by some hopefully deterministic shell scripts.
Flakes unify the two in a common interface. You have the "channels"(called "inputs" in flakes) pinned and the "configuration.nix" (now one of the outputs) pinned together.
That being said, you don't need to switch your system to a flake-managed one right away. If you enable the flags from that article, you can run things from flakes using "nix run". This will transparently pull the flake to your machine and evaluate the output you wanted.
That's where I am in my slow transition to flakes. Still basing my system on a monolithic configuration.nix, but with flakes enabled so I can experiment with them or use them in dev projects as opportunities arise.
One day I'll get around to fully transitioning over to flakes, but it's nice that there's this intermediate step and no rush.
Similar to how package-lock.json records the exact version of packages used to build a nodejs project, a flake lockfile will record the exact version of nixpkgs (and other nix dependencies) the flake uses.
The other thing a flake.nix file provides is a consistent entrypoint to a codebase. Previously, by convention, projects might have a `default.nix`, but one default.nix might be a different form than another default.nix.
The consistent entrypoint allows for a really nice UX for both referring to other nix codebases, as well as a Docker-like UX for running code which has a flake.nix.
With a project specific flake you can specify not only how to build your package, and how to run it, (includable ability to specify "apps" for each executable of a multi-executable package), but also specify a development environment for developing on the package with the exact compiler versions and everything this package wants. (The convenience of a development container, but without limitations of developing in a container.)
This can obviously be extended further to defining nixos modules, which could be useful if your package is a service that people might want to pull in to nixos.
--
Some flake features like defining nixosConfigurations or homeConfigurations are not really sensible for a package-specific flake but can be useful if defining a flake for say your lab of NixOS systems. That is one advantage over classic nixos configurations. You can have a flake that defines the configurations of multiple systems.
Indeed, even if you don't want one master flake for all your machines, you may still want a separate flake for your a home-manager configuration that you can pull in when defining a specific system in a system-specific flake.
Perhaps "Docker, without containers" would be a closer way to convey what Nix does.
With Docker, images get built in a fresh/sandbox environment, and the resulting image is easy to distribute to other systems running Docker.
> I kind of hoped that the Nix language is more sane. I think there is a project of similar goals but the script is in Python?
A project quite related to "Nix (but not using a bespoke DSL)" is Guix, which uses Scheme.
Perhaps you're thinking of Bazel? Which uses a Python-like language, and (AFAIU) is much closer to "like make, but for the whole dependency graph".
Docker, on the other hand, was introduced to solve the deployment problem --- it provides light-weight isolation so that you get boggled in resolving dependency conflicts. Of course, people later also use it drive development environments, which I find quite sensible.
I just started learning about nix a few days ago. I'm still a little bit confused and flakes are an abstraction, i didn't tackle so far.
My mental model is, that it resembles sth. like virtual environments for python or nvm for node.js installations. At least that's what i want to use it for.
Great work from the Determinate Systems team!
I would be interested in it if it didn't require github and github-specific configuations like github action workflows.
In the meantime, you could use a GitHub action to clone your repository from another forge and publish it, if you wanted to.
Just want to put in a request for SourceHut support in case it's not already on your list. Sometimes it gets overlooked in favor of the big corporate forges.
I see in your docs that there is a repository config option, so that could work. If directory works with a submodule or repo pulled in previous job, that could also be an option. I'll give it a try later.
(Also, the support(?) phone number at the bottom seems pretty unusual!)
Just trying to get my head around, as a Nix-interested non-user (other than toying with it briefly years ago) what Flake is, who it's from, how important the controversy mentioned elsewhere is, if I should care, if I'm just starting whether I should use it or not, etc.
Ok Zoomer. (Sorry couldn't resist)
So, old phones had letters on the digits (1 = ABC, 2 = DEF etc.) so you could have the phone number 555-GET-BEEF (555-438-2333) or whatever.
Christ I feel old.
This is to make sure that releases stay available in the long term, even through GitHub account renames or repository deletions.
They’re behind the (I believe to be original) Nix paper back in 2004: https://edolstra.github.io/pubs/nspfssd-lisa2004-final.pdf
Nix's hashing of inputs solves a different problem. It wants to make sure you can have slightly different builds of packages available to the system.
> ...since `patch` updates do not change observable behavior...
Sure, the social guarantee of semver isn't enough to guarantee reproducible packages the same way Nix does, sure. But that doesn't mean that someone depending on v1.1.1 of a package would be happy to suddenly see v2.0.0, or that they'd be unhappy to see v1.1.2.
nix has plenty support for referring to data via git tags, seems like a flake-level semver solver could be implemented locally — even without blessing from mainline `nix` — so long as the flakes you use label their inputs/dependencies accordingly.
Not saying it's a totally unique experience to Nix. It just seems especially concentrated at the moment.
[1]: https://discourse.nixos.org/t/introducing-flakehub/32044/3?u...
If that's the case they may see this as some easy duct tape over top of flakes that could take the pain off of flakes itself while flakes actually work through the process of stabilizing.
[1] https://discourse.nixos.org/t/introducing-flakehub/32044/7
- Nothing we’re doing is inherently proprietary or "secret sauce". FlakeHub isn’t doing anything magical, and the version solving feature would be better if Nix did it itself.
- Also, proprietary services can serve important functions in OSS communities. They can expand the user base significantly (like GitHub to Git) and they can send signals to maintainers of official tools about what’s a hit and what’s a dud. And I think that that’s precisely what we’re doing not just with FlakeHub but also with the Determinate Nix Installer, the Magic Nix Cache, the Flake Checker, and others.
But I think with the general pain and anxiety many longstanding community members are feeling over increasing commercial investment in Nix as well as the whole design and rollout of flakes, more upstream-first development would go a long way (as would quickly making good on statements of interest to upstream these features).
I won't try to persuade you one way or another on commercial investment other than it is here with or without DetSys, and we've released a rather large suite of tools to make using Nix more pleasant and all but one (flakehub's backend) is free and open source. I feel that is a good way to go!
I also think developing features and improvements to Nix outside of it is a good way to prove out ideas that can be eventually merged in to core. We don't _want_ to be solving the version on the server, but I'm rather sure it would have taken months if not years to get an experiment merged. And for good reason: we all want to make sure what goes in to Nix is solid.
I think it definitely can be. Naturally, more DetSys code and ideas that go through that whole journey, the more likely Nixers will be to reflexively trust the company when it comes to new projects. I'm optimistic that that can turn out well.
> we've released a rather large suite of tools to make using Nix more pleasant and all but one (flakehub's backend) is free and open source. I feel that is a good way to go!
I think open-core and proprietary services based on F/OSS can be an effective business model that helps get valuable open-source work done.
For me, Red Hat's model and Qt-style dual licensing are even more attractive because they are based on comprehensive F/OSS licensing.
That said, companies have to figure out licensing strategies that make sense for them and the particular markets and software they're dealing with. I don't begrudge, e.g., Determinate Systems or Cachix or Hercules taking up strategies that involve proprietary services because I've seen people involved with all three companies continue to make important contributions to Nix, Nixpkgs, and F/OSS tooling in the ecosystem that's about more than just plugging into those services. That signals good faith engagement to me.
But seeing stuff like version management get handled in bespoke ways in proprietary services like FlakeHub or at Flox when it remains unaddressed in Nix itself naturally raises an alarm, though.
> We don't _want_ to be solving the version on the server,
I feel like that's good sense architecturally.
Seeing this problem solved in Nix itself will help establish a record and reputation.
Once in a while, there have been an announcement like this, which has not been well received. I am a little worried about it, and what it can lead to for the future. I worry it can split the community and be harmful to the future of very innovative software. I am trying to think of a similar situation within another area of the Linux or Open Source space, but can't really come up with good examples.
Interested to hear if these thoughts are well founded or just a bad take.
I’m no Nix expert but I’ve used it enough to know that the hype exceeds the capabilities and especially the UX of the tool currently. (Just as an example, pinning versions that aren’t in the current nixpkgs release isn’t even possible right now without resorting to overlays and other hacks.) Frankly I’m glad that people are working to make it more user friendly. It reminds me a lot of when I first tried Stack for Haskell: packages just worked out of the box, no more tracking down arcane cabal-install errors and figuring out why it didn’t build in my machine.
That installer has actually been developed with community collaboration from outside DetSys from early on in its lifecycle via the Nix Installer Working Group¹, which includes stakeholders from other companies and whoever else volunteered to get involved.
I haven't followed it super closely recently, but to the best of my knowledge DetSys has done a good job with that project of orienting themselves towards the wider community and upstreaming that work.
FWIW, the maintainer of and chief contributor to that project is also extremely responsive to community activity on GitHub as well. I reported a few bugs the other day and was frankly delighted by how it went. It's being developed in the open, like proper open-source. The Determinate Nix Installer is not at all a 'let's do this in-house, behind closed doors, then throw it over the fence' kind of effort.
--
1: https://discourse.nixos.org/t/nix-installer-workgroup/21495/...
We think it's perfectly okay to have that kind of distribution of labor in OSS, where core/upstream things need to be extremely stable and conservative while the community runs experiments and figures out what works and doesn't.
[1]: https://determinate.systems/posts/determinate-nix-installer#...
> We think it's perfectly okay to have that kind of distribution of labor in OSS, where core/upstream things need to be extremely stable and conservative while the community runs experiments and figures out what works and doesn't.
I hope nothing I said suggested I didn't think this was ok. I really appreciate that the DetSys installer exists. I'm just not sure why anyone would use the default installer given that it does.
Nix has a higher difficulty curve, and rougher corners, than most tools out there.
The onboarding experience can vary from "wow that was easy, this is incredible" to "this is easy to do with Linux, but why is this difficult on NixOS?".
Nevertheless, Nix is a much more powerful tool for dealing with packages compared to what's widely used today. I don't think "overhyped" is deserved, even though someone new to nix will likely have difficulty trying to use it for everything.
> Just as an example, pinning versions that aren’t in the current nixpkgs release isn’t even possible right now without resorting to overlays and other hacks
I read this as "package foo v1.1 was in nixpkgs at some point, but's now foo v1.2". Not sure I'd use the term "hacks", but there are several ways you'd be able to refer to a v1.1 package, depending on what you're trying to do.
But, yeah, if you want a different version than one that's provided by someone else, you're going to have to come up with something yourself.
Yeah, this is the part I find a bit silly. There should be one way to do this. What I want to do is pin an Erlang version (down to semver patch) and then pin an Elixir version that’s compatible with it (also to semver patch). This is trivial to do with my existing tools (asdf/rtx and Dockerfiles), but it doesn’t have first-class support in Nix. I have to build that capability myself! I was absolutely floored by this after hearing so much about how seriously Nix takes reproducibility.
Similarly, the default Elixir and Erlang packages don’t even have a version, you have to use one of the special elixir_1_14 packages to get a specific minor version. The default Elixir just gets the latest release at the time nixpkgs was released. I can pin nixpkgs trivially but it’s not intuitive to pin individual packages across nixpkgs releases. Clearly this is just how Nix works, but because it’s not how any other dependency managers work, it’s a lot harder to integrate piecemeal.
edit: to Jetpack’s credit, this is one of the things I really like about Devbox. It makes pinning minor versions easy!
Note that this approach isn't without its downsides (compared to overriding the version in nixpkgs, or using a separate nix package). -- Mainly, since it uses an older nixpkgs version, the dependencies will also be that old version. (They won't have any security fixes which have been released since then; and it'll take up more disk space to install).
> Similarly, the default Elixir and Erlang packages don’t even have a version, you have to use one of the special elixir_1_14 packages to get a specific minor version.
Supporting multiple versions be more helpful at the cost of more effort to maintain. It's a reasonable tradeoff that nixpkgs provides the latest, and if you want anything other than what nixpkgs provides, you write some nix for that.
Correct.
> It's a reasonable tradeoff that nixpkgs provides the latest, and if you want anything other than what nixpkgs provides, you write some nix for that.
Reasonable for you, maybe. For me, I have dozens of Erlang projects on various versions that I don’t want to break to start using Nix, and Nix is significantly more effort to get a reproducible configuration than my current setup. Why bother? In what universe is it reasonable to make package consumers write software to consume different versions of a package? Tools are supposed to make our lives easier, not harder.
Docker is also a tool where the UX is nicer if you write code (e.g. a Dockerfile) rather than using cli only (e.g. `docker commit`).
I don't think there's anyone who would prefer to use `docker commit` instead of using a Dockerfile.
With Nix, it's similar: only using Nix through the CLI, and not writing Nix code, is going to limit how useful Nix is.
In any case, you don't need to write Nix code to use a package from an older revision of nixpkgs. -- With different options, there are different trade-offs.
> Nix is significantly more effort to get a reproducible configuration than my current setup. Why bother? ... Tools are supposed to make our lives easier, not harder.
Yeah, Nix is much more difficult to use than other tools.
Nix's UX is one where you pay a big cost upfront in order to save on effort later.
e.g. that's comparable in Infra-as-Code, where writing Terraform is going to be slower than just managing stuff through a graphical console.
Here's proof:
Is it possible to write C programs that manipulate strings? No, because string library functions are "hacks." Is it possible to write a bash script that passes arguments with spaces to a command? No, because quoting and character escaping are "hacks."
Nix packages are malleable. With a bit of code, customizations can be applied to individual packages and entire dependency closures alike. An overlay is an embodiment of this property. It allows the user to create packages that fits their unique needs on the fly without having to repackage the whole world. No other package manager maybe besides Guix has this capability. I don't think it's fair to dismiss it as an "hack."
But as good as flakes are, they still have big problems. flake.lock size explosion, UI hassles, no cross compiling support.
In addition, we're invited to join the discussion on Discord, of all places, instead of the other two standard messaging platforms that the Nix community typically uses (Matrix and IRC)
Eh, they had a pain-point, built a tool to alleviate it, and shared it. I suppose the less tone-deaf alternatives are "keep feeling the pain (by not building it)" or "let everybody else keep feeling the pain (by not sharing it)." I can't complain too much with that.
> In addition, we're invited to join the discussion on Discord, of all places, instead of the other two standard messaging platforms that the Nix community typically uses (Matrix and IRC)
100% with you on this point.
I'm getting the hint that some people aren't happy with the current state of flakes. But right now it's the best solution for a certain large class of problems so it's what people are going to go with. Again, as a relative outsider I see flakes as The Way Nix Works with the requirement of enabling experimental features just being a part of the Nix process. Nix is rapidly growing in popularity with a lot of people drawing the same conclusion as me. Maybe there is a better alternative to flakes in the horizon, and when it's ready I'll consider switching. But I'm not going to wait to use Nix in the meantime, and for me the best way to use Nix currently is flakes.
In other words, yes: we are doubling down because we don't see a future of Nix where flakes or something like them aren't central.
Most of our tools are open source, totally free, and don’t depend on us as a company continuing to exist. In many cases I hope the Nix project wants to adopt and take over some of our work — like the installer.
Our goal is to make Nix as good as possible, and do it together. In the meantime, it’s okay to not use our tools :)