Nickel Modules | Hacker News Reader

Nickel Modules

tweag.io

57 pointsryangibb21 comments

There's no explanation of what Nickel is. Even if I click the link, I get:

>Better configuration for less

>Write complex configurations. Modular, correct and boilerplate-free.

I still don't get what this is. Configuration for what? Is this an alternative to TOML/YAML?

From clicking three or four links deep into the Nickel documentation, I finally found an explanation that Nickel is a language for generating configuration files like YAML or TOML.

Either I'm super out of the loop or the authors need to do a better job of explaining what Nickel is from the beginning.

(disclaimer: Nickel dev)

Thanks for the feedback. It sounded obvious to me, but I reckon that after looking at the website again, it's never spelled explicitly really on the front page.

Nickel is a domain-specific programming language for configuration. Similar to CUE, Dhall, Pkl, Jsonnet or the Nix language. With typing and validation capabilities built in. I like to think of it as "configuration templating done the right way". While it originally targeted Nix in particular, it's a standalone config language that can be used to generate YAML or the like for e.g. Terraform, Kubernetes or anything where using pure YAML (or ugly unplanned semi-programming languages disguising as YAML) isn't enough.

Is this by any chance reference to the nickel (NCL) config language at google? Otherwise a fun naming coincidence.

Fun story, google had a ton of GCL configs (author of CUE worked on GCL) which has certain unfortunate semantics that can't be fixed without breaking compatibility. NCL was designed to address those but of course automated migration wasn't possible due to tricky semantical differences.

There was a big push to move to NCL but it failed just due to sheer size of GCL codebase and was abandoned and rolled back, and NCL was deprecated. Unfortunately some new tools already picked up NCL and then had too much code to go back. So to this day most configs are GCL but there is one tool that remains on NCL.

Actually we learned about Google's NCL way after, when discussing with CUE's author, that we happen to know as well. So it's a funny coincidence. Nickel originated from NCL which is both a recursive acronym (Nickel Configuration Language), but also came from the ambition of improving Nix: it also meant, at least initially, Nix Configuration Language, I believe. Maybe there's been a wordplay around Nickel <-> cheap configuration language as well.

We also co-organize CONFLANG (https://2023.splashcon.org/home/conflang-2023) with CUE's author and other people, and we got several subscriptions and talks from Google last year. This is available here: https://www.youtube.com/watch?v=NbEkN8OOQFQ&t=23367s. I you look at the first comment, all talks are indexed, and there is in particular an experience report on improving GCL incrementally without breaking anything at Google (https://www.youtube.com/watch?v=NbEkN8OOQFQ&t=23367s). Very interesting!

Thanks for the comprehensive reply. The resources you linked look great, I'm putting that on top of my watch list.

YAML is the bane of my existence and I daydream about building a purely declarative system for running services with a sane config language (CUE or Pkl mostly). But then reality comes back and I have to spend another hour adding nindent to text based yaml templates...

I've always imagine the name came from a thought like

> If I had a nickel for every time someone said 'Nix would be perfect if only it had a good type system...', I'd buy a better configuration language!

but I completely made that up in my own head :)

Cool, thanks for the additional context!

As someone who is a Nix beginner and struggles with the language, I'd love to see an alternative that's easier to use.

Is Nickel at the point where it is a viable alternative for Nix than the Nix language? Is there any documentation about configuring Nix with Nickel?

I think the answer is yes and no. As a language (and as far as tooling is involved), I think Nickel is entirely suitable to fill the same role as the Nix language (maybe at the same scale as Nixpkgs the performance will need some work, but that's it).

Unfortunately, being a Nix replacement involves much more than just being a good language for configuration. It requires to interact tightly with all the existing Nix code that has been produced (basically, Nixpkgs). This is the hard part, which isn't there yet. Some design document that was started a long time ago can give you more details: https://github.com/tweag/nickel/pull/693.

The best current take is organist (https://github.com/nickel-lang/organist), which is much more limited in scope (basically development shell), but has some form of interaction with Nix and Nixpkgs, albeit limited. I think the recent development around json-schema-to-nickel and the availability of JSON schemas for NixOS modules (as mentioned at the end of the post) might make it realistic to write boring NixOS configs in Nickel (and have the nice in-LSP validation), but we haven't had the time to try this combination yet.

nickel's performance definitely will need some work. It's several orders of magnitude slower than nix for even simple tasks:

    $ time nix eval --expr "builtins.foldl' (l: r: if l > r then l else r) 0 (builtins.genList (x: x) 5000000)"
    0.839s
    memory: 627 MB
    $ time nickel eval <<<"std.array.fold_left std.number.max 0 (std.array.generate (fun x => x) 5000000)"
    1:20.06s
    memory: 10540 MB

I know we don't actually need to deal with 5 million element lists in practical code, but this is also not nickel's most pathological case, and it's easy to write fairly reasonable and normal code which is prohibitively slow.

I rewrote some code I had laying around from nix, where it evaluated in a mere 3s or so (using perhaps a GiB of memory), into nickel.

At first the nickel version crashed after using all my memory, but with hours of optimization, I managed to get it to run in just under 2 hours with only 60GiB of memory usage.

That looks bad indeed.

I must say that contracts, while very nice for error reporting and all, also brings additional challenge related to performance (cf https://dl.acm.org/doi/10.1145/2914770.2837630). Our stance is that it's a reasonable trade off for configuration, but the cost would be unacceptable in a general purpose language.

Also, it's true that we've focused on language design and tooling before performance, and then working our way when users report unacceptable perfs on their concrete use-case.

There's been some improvement recently-ish. For example, with the latest Nickel, I get (by inlining `max` to mimic the Nix version):

    $ time nix eval --expr "builtins.foldl' (l: r: if l > r then l else r) 0 (builtins.genList (x: x) 5000000)"
    0,74s user 0,11s
    $ time nickel eval <<<"std.array.fold_left (fun x y => if x > y then x else y) 0 (std.array.generate (fun x => x) 5000000)"
    36,63s user 3,40s

Nothing to be too excited about, though. Although it doesn't make it unusable in small and medium-sized projects (we have users with 120kLoc of Nickel - with they'd definitively like to make it faster, it's still usable), I agree that performance is a big area of improvement, to say the least.

(also, it seems `foldl'` is an ad-hoc builtin operation in Nix, probably for perf reasons, so the comparison isn't entirely fair either)

If you don't like the Nix language, you're unlikely to like Nickel - they're very similar.

I wouldn't say they are very similar. Nix is a fairly barebone functional language, so it does embed fairly straightforwardly into other functional languages (including Nickel, but it's true for Haskell, OCaml, Dhall, Jsonnet, Scala, etc. as well - forgetting about Nix-specific builtins, as long as you have higher-order functions, records and arrays, you have the Nix core).

However, Nickel adds pattern matching and destructuring, reverse application, types, contracts and merging, which I think does lead to a quite different experience in practice. It's a bit like saying that C is similar to Rust (ok, so the analogy doesn't work as well for Rust as most C would be unsafe on non-idiomatic Rust) or C++: in some sense, yes, but developing in those languages is very very different.

In Nix, I've found that the biggest problem is not the language but the lack of easy introspection. Which keys are present in the struct I'm looking at? What are all the symbols in point X? Etc. It felt like an "ok to read but hard to write" language.

I think it's resolved by a proper IDE and runtime-inspection of some kind.

I would weaken and expand this a bit.

1. If you like Nix, you'll probably like Nickel, too.

2. If you use Nix enough that there are enough specific things you hate about it to comprise a wishlist, you might be extra excited about Nickel for checking some of those boxes.

3. If you looked at Nix code for 5 minutes and instantly bounced off it because it seemed alien and unpleasant, Nickel might give you similar feelings.

I think (3) often resolves itself with experience, so it's potentially worth coming back to Nix or Nickel in a more patient and curious mode, if you've had that reaction.

Same. It was only when I got to reading the user manual that I see what I feel could be the first sentence of their website

> Nickel is a generic configuration language.

> Its purpose is to automate the generation of static configuration files - think JSON, YAML, XML, or your favorite data representation language - that are then fed to another system. It is designed to have a simple, well-understood core: it is in essence JSON with functions.

https://nickel-lang.org/user-manual/introduction/

Nickel is a replacement for the Nix programming language

I wonder how much Nickel has improved since last year. I attempted to use it to generate some Terraform code, and the experience was less than pleasant. This was largely due to surprising interactions and lack of documentation, ironically making it similar to Nix.

I'm hopeful it will keep improving.

i bet this nickel language is awesome and versatile, but that their own employees have hidden bash and python hacky scripts that generate what amounts to 90s-Oracle-consultant-xml levels of verbosity for simple config files just so you can have a test db for ci builds in the end.

Coming from Rust, there's still something about those Nickel error messages that don't read as well for me. They're much better than Nix for me, but weirdly they still read a bit busy. Maybe it's just my brain heh.. i can't explain it.

Nonetheless I still am just dying to switch to Nickel from Nix though. I am so tired of not having types in my config language, crappy documentation available to my LSP, etc. The wait for Nickel has been long for me

Nix improved error reporting around version 2.20[1]. Specifically the dreaded infinite recursion can now be traced back to the code that caused it without poring over the whole stack trace.

Contracts in nickel are much better IMHO.

1: https://nix.dev/manual/nix/2.22/release-notes/rl-2.20

Reply on news.ycombinator.com