Improved evaluation times with pre-resolved Nix store paths | Hacker News Reader

Improved evaluation times with pre-resolved Nix store paths

determinate.systems

41 pointsbiggestlou48 comments

DetSys, once again, building a moat.

I really do not like how this approach depends on their seemingly not open source backend. Nix needs tools like this, no question, but I'd hate to see it get adoption through a way that isn't reproducible by anyone. It looks like it could be a repeat of the Snap store problem - interesting tech, made unusable through de-facto dependencies on proprietary services.

I didn‘t look too deep into Nix the last couple of months (> 12) and was wondering while reading: what the hell is fh now. Another abstraction? I share your views here!

fh is the CLI tool for FlakeHub. It's been around almost as long as FlakeHub itself. It's pretty standard fare, not really an abstraction per se.

‘Standard’ if you live in the flakehub ecosystem, which the vast majority of Nix users don’t.

Standard in the sense of "a non-magical CLI tool that wraps a platform's HTTP API," not standard in the sense of "used by a majority of users."

They're not building a moat, they were excommunicated and banished. Zealots hate to see heretics living successful lives. More power to them.

I think you may be mixing up some things. Anyway, where's this sudden influx of traffic coming from?

>I think you may be mixing up some things.

I don't think so.

>Anyway, where's this sudden influx of traffic coming from?

What traffic?

Out of curiosity, do you object to the existence of services like Cachix, Nixbuild, and Garnix, which also do not have open source backends?

Is that a fair comparison?

FTA:

> Even if you can (ideally) pull a path from a cache instead of building it, you still have to pay Nix’s evaluation tax in determining the derivation’s store path in the first place. [...] Fortunately, FlakeHub now offers an elegant solution to this problem: resolved store paths.

If I may paraphrase:

> [fundamental problem with Nix] -> [proprietary solution].

Cachix and nixbuild feel like they're solving something at a different layer. They fit into the existing Nix ecosystem (binary cache, builders) and just provide a hosted version of an existing concept. They don't change anything fundamental about Nix.

Maybe a better comparison would be devenv?

I don't object to the product, I object to DetSys moving CppNix into more of an de-facto open core product[1]. This is just a symptom. DetSys contributes the majority of the code that actually gets prioritized and merged (because the maintainer is a DetSys employee), but asking them to build better caching mechanisms into CppNix (which, to be clear, I never asked, that's just Luc's strawman of me) instead of their proprietary product is suddenly "asking Eelco to work for free".

[1]: https://discourse.nixos.org/t/announcing-determinate-nix/547...

If their founder was the person with principal control over what goes into CppNix, yes.

Is there a Nix-based company co-founded by Eelco Dolstra to which you wouldn't object?

Sure, but you seem to sidestep the primary accusation that Nix itself got neglected in favor of DetSys. Which is most certainly true for at least the installer.

I don't get this argument. It is clear that if it wasn't for DetSys making a better installer, we wouldn't have one, period.

And now there is some movement from NixOS to bring the new installer as a replacement of the old one: https://github.com/NixOS/experimental-nix-installer. It doesn't seem that Eelco or anyone in CppNix is actively trying to sabotage the improvement in the installer, but they simple don't care enough to improve the situation (that is fair enough, I assume most in the core team use NixOS).

The expectation seems to be that Eelco work on Nix full time for free lest he be accused of neglect. Is there a way for Eelco to make money that you would approve of?

I would've replied to you but HN's intransparent timeout mechanism hit me.

I'd invite everyone to wonder why Lix exists (and is better enough to be used in some parts of Nix's CI now), why CppNix stable was stuck 6 versions behind on nixpkgs until half a year ago, and what kept Lix contributors from working on CppNix instead. Tip: "I want free labor from Eelco" is not the answer.

Nobody said this. I suggest that you not escalate the discussion in this direction. You're obviously biased given involvement with detsys, nobody expects you to think differently. This crass dismissal off the issue with a strawman isn't right.

Love what they are doing. At least you get the chance to introduce Nix in the enterprise with the MacOS installer, having figured out private CAs and the MacOS keychain for example. Then MDM.

Is it really a moat though? The same flake can still be evaluated without this service, it just takes longer.

And I can still install a snap on an Ubuntu Core machine by building my own distribution method, but it will be a downgrade from the experience of running "snap install something" that introduced me to it. So nobody does.

Lix (yes, I will mention it again) essentially exists by a large degree because the overlap between the people that control Nix and DetSys is big, even very big if you look at CppNix.

I don't follow this argument. Unless you chose to step in to DetSys ecosystem with this fh tool or whatever, this has no impact on you at all.

> exists by a large degree because the overlap between the people that control Nix and DetSys is big

No, it exists because of sexual identity politics. 11 out of 12 of the Lix developer team are transsexuals. Clearly the selection isn't about technical issues.

Who's doing the identity politics now.

Clearly the Lix team.

Ok but they’re creating a good product, in their “own” space, not bothering anyone, and they increase competition which benefits the entire ecosystem, so to be frank… who cares?

Lix is a net win for everyone. It would behoove us to stop taking pot shots at them because if you want nix to succeed, the presence of Lix is a net positive.

Not sure deliberate and obtuse fragmentation benefits anyone.

If they started to fix some of the long-standing Nix implementation issues (namely, evaluation performance) that would be one thing. But what they actually did is rage-fork nixpkgs and register new domains.

>Ok but they’re creating a good product,

Debatable.

>in their “own” space, not bothering anyone, and they increase competition which benefits the entire ecosystem, so to be frank… who cares?

Sounds like White supremacy and Neo-Nazism. Why not say that about every other project that leftist enter and try to push their religious beliefs?

>Lix is a net win for everyone.

Most definitely not, fragmentation and religious proselytizing destroys projects.

>It would behoove us to stop taking pot shots at them

The truth shouldn't be hidden, regardless if they're "pot shots" or not.

>want nix to succeed, the presence of Lix is a net positive.

The Lix team are the ones that destroyed the nix community with their totalitarianism, leftist/woke ideology, and subversive actions. They are a net negative.

Nobody is keeping cis people from contributing to Lix, and even if they did, it'd be far from them trying to eliminate you. Isolationism isn't genocide, you fetid moppet.

Tbh I'm starting to believe that Lix is a great litmus test to determine if someone was honest about "not caring" about gender identity.

People who opposed the introduction of gender politics in mainline Nix, but now also dislike Lix... what on earth do they want? It's starting to sound like they just want transgender people to be unhappy.

To your parent poster: you got what you wanted, "they" are "gone". What are you afraid of? That they're better? Even if they did reject contributions from cis people: let them! What if transgender people are inherently better programmers and we hold them back? What an amazing way this would be to find out.

I disagree with the nazi comparison but I will leave it at that. :D

(Inb4 "fascism")

This is cool! We do something similar in Devbox where we pre-evaluate store paths from the Nix repository, and then fetch the packages directly from the official Nix cache. This greatly reduces the evaluation time.

Here's a blog post we wrote on the topic:

https://www.jetify.com/blog/how-we-sped-up-nix-package-insta...

We also do something similar at garnix, but when enabling incremental builds. Instead of just skipping the build stage, we also “normalize” the eval into just the store path, and skipping it the second time around.

Mentioned in passing in https://garnix.io/blog/incremental-builds. This is even more significant because in this case you might otherwise be eval-ing several layers of flakes.

Blog post author here! Please feel free to ask me any questions directly in this forum :)

Can the evaluation tax be quantified in some way? In other words, how big is the problem that this solves?

Unfortunately, it's really hard to generalize. If the evaluation involves Nixpkgs, for example, Nix needs to copy Nixpkgs into the Nix store, which is many MB. And of course this needs to happen for all flake inputs. I've seen this evaluation take a few seconds and I've seen it take several minutes, but in general the more flake inputs the longer you can expect it to take.

> If the evaluation involves Nixpkgs, for example, Nix needs to copy Nixpkgs into the Nix store, which is many MB.

Only if you use flakes (and this behavior is a good reason to not use them).

Using flakes or not, any store path evaluation involving Nixpkgs is going to be computationally intensive, potentially too intensive for a memory-constrained device.

(DetSys Cofounder here) I've had Raspberry Pi's give up their last ghostly breath evaluating NixOS.

Is this similar to, using nixos-rebuild and target a second machine?

I use my laptop's nixos-rebuild and push a new generation to another device. The 2nd device doesn't have the config, so it won't be able to build it. For these devices, I don't want the config to be on the 2nd device.

This will make discourse.nixos.org explode again. It happens every time Determinate System publishes something.

The unusually high amount of drama in NixOS is one of the reasons I have been hesitant to start using it. Jon Ringer (a former contributor) posted a giant video [0] outlining his history with the project and how he left / was forced out (amongst several other key folks). For the record, I don't know how to characterize his exit, only that there was an absolute ton of drama around it that wasn't really related to even any technical aspect.

[0] https://www.youtube.com/watch?v=gp0FI8Gw1iA

I don't agree with Jon on a personal level, but who was technically in the right aside, he also made some choices that'd accelerate your exit from any organization or community during the events.

Let's just say his approach wasn't one of reconciliation and de-escalation.

Listened to this video yesterday and it just makes me sad. In my opinion, Jon has been let down by the community to which he belonged. It is clear there were differences of opinion and so on, but I did not see anything that motivated his expulsion permanently. It left an especially bad taste that it was on recommendation by the Constitutional Assembly. Their mission was to propose a constitution, not to act as arbiter in moderation matters.

So you don't use it because of the drama, you only repost the drama?

What a collective waste of energy. Had no idea any of this was/is going on. For the sake of my personal systems I hope NixOS makes it; wouldn't be the first community that fails because of such kindergarten nonsense...

It is certainly looking grim. The current leadership is more interested in fostering an environment they seem "acceptable" than contributing technically to the project.

Nah - they look like they are too busy having the n billionth argument over telemetry in packages, and the n billionth argument over monorepos.

So this will probably get placed into the drama backlog for a month or so

what a coincidence, my eval times have worsened over the past months too!

Reply on news.ycombinator.com