Hacker News Reader

> 6. Good support for building against external packages that may already be on your system.

Able to use 'internal' packages (spack install openssl) or external packages (link against /usr/lib/libssl.so) on a per item basis: sometimes you want the newest code, but other times you want the OS package so that you can get security updates 'for free'.

Most other system do one or the other: Spack allows you to choose.

What do you think about Spack vs. Conda/Mamba? I've been a contented (maybe not "happy") Conda user for years. Is there any reason to switch? I'd be concerned about package availability.

Spack has Anaconda as a package:

* https://spack.readthedocs.io/en/latest/package_list.html#ana...

* https://spack.readthedocs.io/en/latest/replace_conda_homebre...

So you can continue using it, but you also have a super-set of features for other things as well.

We have a guide for this written by Adam Stewart, probably our most prolific (package) contributor:

* https://spack.readthedocs.io/en/latest/replace_conda_homebre...

The thing you'll likely notice most about Spack vs. conda/mamba/rattler(?) is that it lets you pick your versions and options much more fluidly, and it'll build you a package from source if there's no binary for it. Spack is much more seamless for integrating binary installations with source builds. `conda-build` is a whole other tool outside the normal user workflow.

Spack has over 7,000 packages at this point, so I don't think the lack of packages is going to hurt too much. We don't offer nearly as many public binaries yet, though, so the source builds may make the UX more painful, depending on what you're trying to do.

Very neat:) Given the similarity in features, could you comment on how it compares to nix?

+100

From what I saw the package definitions are in python. Given python is really an imperative language I don't see how spack could be comparably powerful for definitions and reuse.

The package definitions are quite declarative even though the language is imperative, see an example here [1]. For hashes, Spack uses the normalized (comments, whitespace etc removed) abstract syntax tree.

With Spack it's easier to contribute, you don't have to learn a new language as you need for Nix / Guix.

[1] https://github.com/spack/spack/blob/develop/var/spack/repos/...

I could be persuaded to trade a certain amount of power for ease of use

The Spack DSL for defining versions, options, etc. is declarative. That part of a package exposes options to the solver and very much allows reuse, in the sense that the same package can be built many different ways, and even with different dependencies. The whole system is parameterized in ways that Nix is not.

The imperative part of a Spack package is the build recipe, which AFAICT is not so different from bundling a bash script in a Nix derivation. Just like Nix, it's included in our configuration hash, so if you change the recipe you get a different installation of the software.

Thanks for the answer.

> That part of a package exposes options ... in the sense that the same package can be built many different ways, and even with different dependencies.

It is quite a common pattern in nix to have package build flags and the dependencies as inputs in a derivation. This definitely makes it possible to build in different ways and with different dependencies.

Yep! I'm not a nix expert but my understanding is that this is typically done with overlays. So you can, say, swap out openmpi for mpich, but mpich requires a different version of hwloc that conflicts with the one defined in some other branch of your derivation, you're out of luck. In Spack that is handled through dependency resolution. There have been discussions of how to deal with this in Nix, e.g.: https://discourse.nixos.org/t/concept-use-any-package-versio....

Similarly, for packages like HDF5 that tend to be depended on with options that affect its API (+parallel, ~parallel, etc.), I do not think there are automatic ways to get nix to ensure that your request for hdf5+parallel is consistent with other packages in the graph.

Possibly worth a mention: for uarch flags like the ones shown here: https://nixos.wiki/wiki/Build_flags, the user is responsible for setting them specifically for the requested compiler. We've abstracted that with `archspec` (https://github.com/archspec/archspec) so that you can just ask for a target and the correct compiler flags are injected. e.g., `target=cascadelake` or `target=zen3`. So if you switch compiler, you do not have to look up these details. See https://tgamblin.github.io/pubs/archspec-canopie-hpc-2020.pd... for more on how it works.

The salient parts of archspec aren't in a python library but in a JSON file (https://github.com/archspec/archspec-json/blob/master/cpu/mi...) so it's something that could probably be incorporated into Nix.

After using it for ~8 years, I rather like Nixlang. At the same time, I don't know Python well or particularly like it, so Spack's DSL just doesn't look as comfy to me. It reminds me of RPM spec macros a little bit, which I guess are fine but I don't love them. I'd rather see Nix learn from Spack than myself jump ship.

But I think it would be very silly to dismiss Spack over something like the fact that their DSL is Python-based.

It's fine to be partial to what you know, and there are lots of good reasons to prefer Nix today, at well as to think that improving Nix might be better for you than adopting Spack.

But I think we should go a bit deeper than this in evaluating competing projects. Nix is about more than just being in some FP club, you know?

Main differences with Nix:

* Spack has a dependency solver (concretizer). nixpkgs is managed by humans who do the dependency solving for you over time. What this means practically is that you can install 5 versions of some package with different dependencies and flag combinations with a few commands in Spack, while with Nix you would probably need to check out different commits of the nixpkgs repo and maybe do some hacking on nix derivations to get the same thing. Spack is fundamentally designed to help with combinatorics.

* Nix builds all the way down to libc; Spack (currently) doesn't -- it's designed to live on an existing system.

* Nix build environments are isolated (but require root to run); Spack's aren't as completely isolated and don't require root (so you can run Spack in your home directory.

* Spack installation hashes are what Nix would call full configuration hashes -- they're configuration metadata hashes, not content hashes of the installation. So you could say we're not quite as committed to exact binary reproducibility, but you could also say that this allows us to support relocatable binaries (which Nix does not). Also, our metadata is pretty detailed.

* Spack's DSL is Python; Nix is, well, Nix.

Spack is very much inspired by Nix, and we talked about this a bit in the original paper here:

* https://tgamblin.github.io/pubs/spack-sc15.pdf

> with Nix you would probably need to check out different commits of the nixpkgs repo and maybe do some hacking on nix derivations to get the same thing. Spack is fundamentally designed to help with combinatorics.

Typically you'd use the overlay mechanism combined with package overrides for this, so you would have to write some Nix code, but you can do it out-of-tree (the most common location of choice would be inline in your configuration repo).

One of the usability issues with this is that the APIs for overriding dependencies and flags in Nix packages are sometimes a bit language ecosystem-specific. The uniformity of Spack's approach strikes me as an even bigger win over Nixpkgs here than the fact that you can append those tweaks directly to CLI invocations without saving any code to a file. :)

> Spack installation hashes are what Nix would call full configuration hashes -- they're configuration metadata hashes, not content hashes of the installation. So you could say we're not quite as committed to exact binary reproducibility, but you could also say that this allows us to support relocatable binaries (which Nix does not).

Nix supports content-addressed store paths but it's still not very widely used yet, and it's opt-in. A normal system with the feature enabled will have a mix of configuration-addressed and content-addressed paths.

Spack looks really, really cool. Thank you for advancing the state of the art. :D

I hope Spack's package collection is seeing and will continue to see exponential growth like Nixpkgs and Guix are! It seems like the stronger design fundamentals in these newer package management ecosystems can really enable that.

> nixpkgs is managed by humans who do the dependency solving for you over time

One nice feature of Nix is the "import from derivation" pattern: if some Nix definition happens to call the built-in functions 'import' or 'readFile' with a path to some other Nix derivation (AKA build product), Nix will first build that derivation, then import the resulting file to finish evaluating the original definition.

This way, we can have one derivation which runs some arbitrary dependency-solving command; and import its result as part of our package's definition. We do this where I work: we run the `mvn2nix` command inside one derivation, to get a JSON file describing our Maven project dependencies (jar and pom files); this is imported, and used to define a self-contained folder of those jar and pom files; then the main project definition runs Maven pointing at that folder.

The downside of this approach is that it slows down evaluation: Nix usually has an "evaluation phase", where we figure out what to build; then a "build phase", which runs those builders. When we "import from derivation", this distinction is lost: in order to figure out what to build (e.g. which Maven command to run, to build our project), we must first do some building (e.g. to figure out what should go in the folder that Maven will look in for dependencies). For this reason, the Nixpkgs repo doesn't allow definitions which use "import from derivation"; however, it's a very handy tool for our own personal or organisational projects :)

(PS: Many years ago I tried to do something similar for building Haskell projects; but never really got it to work :( )

I'm not sure if I understand the first point. Doesn't Nix currently achieve the same thing feature-wise?

For example,

    spack install packageX %libA@2.0

would translate to

    nix-build -E 'with import <nixpkgs> {}; packageX.override { libA = libAv2_0; }'

For this to work, both Spack and Nix would have to package libA v2.0. If it isn't, the user would need to create their own. Assuming a different version of libA is already packaged in Nix, here's how it would look like without the "nix-build -E" part:

    let
      pkgs = import <nixpkgs> {};
      libAv2_0 = pkgs.libA.overrideAttrs (old: {
        src = pkgs.fetchzip {
          url = "...";
          sha256 = "...";
        };
      });
    in
    pkgs.packageX.override { libA = libAv2_0; }

I assume something similar would be required for Spack too.

So aside from Spack having a nicer shorthand syntax for customization, I don't get what Spack can do that Nix can't in terms of features. Or to be more specific, how a dependency resolver can eliminate human work.

Spack packages are parameterized, so there is one `package.py` file per package, not several, and you can have many versions and options declared. See, e.g., `zstd`: https://github.com/spack/spack/blob/develop/var/spack/repos/...

This was a conscious decision we made to allow many versions to be built without requiring split implementations and without checking out different commits -- that saves some work already. It also forces the repo maintainers to consider the other use cases, which tends (IMO) to make the recipes more portable.

But the real question is not just work saving but correctness. You cannot always just swap in libA@2.0 as in your example. For:

    packageX ^libA@2.0

Suppose that both packageX and libA depend on MPI (a versioned standard for which there are several implementations) and, further, libA requires MPI@3. Spack will ensure that:

    1. packageX and libA use the same MPI implementation (e.g., openmpi, mpich, mvapich)
    2. the MPI implementation chosen satisfies the provider requirement
    3. any version/option constraints that packageX has on MPI are satisfied along with those of libA.

You could also imagine that the two packages might have conflicts with certain implementations of MPI, e.g. say packageX conflicts with mpich and libA conflicts with mvapich. You'd have to choose mpich.

This isn't just an overlay; it's a constraint solve. The choice of libA@2.0 can have effects on other nodes in the graph, and different choices may need to be made elsewhere; maybe even things like disabling options or choosing different dependencies.

Compared to homebrew, do prebuilt binaries have to be installed in a predefined path they were built in or is it also flexible?

Also, is there an indication/warning whether the package is prebuilt before you install?

> do prebuilt binaries rerouted to be installed in a predefined path they were built in or is it also flexible?

Prebuilt binaries don't have to go in a fixed prefix -- we'll relocate RPATHs, shebangs, strings in text files, and (as long as you build in a longer path than you install to -- see `padded_length` at https://spack.readthedocs.io/en/latest/config_yaml.html) strings in binaries.

There is not currently a way to see before you install if an install will be from binary or from source, but that is something we are working on.

So how would I know whether the prebuilt binaries were built in a longer path than I install to?

Is there at least a way to block source builds if the app is not in the binary cache to avoid any surprises (basically have manual override/confirmation instead of automatic fallbacks to source builds)?

> So how would I know whether the prebuilt binaries were built in a longer path than I install to?

All the public ones are built with very long padding; if you make your own you're currently on your own.

> Is there at least a way to block source builds if the app is not in the binary cache to avoid any surprises (basically have manual override/confirmation instead of automatic fallbacks to source builds)?

Yep:

    spack install --cache-only ...

lmod integration?

Yes:

* https://spack.readthedocs.io/en/latest/module_file_support.h...

Both Lmod and (TCL) Modules: can generate either/both.