Clojure 1.13 adds support for checked keys | Hacker News Reader

Clojure 1.13 adds support for checked keys

clojure.org

225 pointsFelipeCortez47 comments

This is a case I never really thought about - if the key is missing today you'll get nil as the value and since Clojure is a nil punning language it usually does sensible behaviour in your program

I know this sounds unreliable but in practise I like a language that defaults to pragmatic code paths so I don't have to stay up at night imagining a million code paths

This adds a throwing codepath which is quite drastic so I'm glad people don't build this into programs everywhere - I'd be nice to hear what the team imagine as the use case for this

Normally for correctness I'd like to see specs at the boundaries for programs and different test suites for internal behaviours

The problem in my experience is that while nil is a perfectly reasonable default 9/10 times that 1/10 happens often enough and causes major problems that it is worth taking the extra few seconds to write it explicitly in the code to acknowledge that case and that you have checked that it is fine in the 9/10 cases or handle it in the 1/10 case.

I have seen multiple major production outages in Golang code because people accidentally read a non-existent map key and used the default value. As a funny bonus in one of those cases we were stumped when debugging because this code had tests, but the tests were also reading the default values out of the map and asserting that "" was in fact a valid textproto (it always is!) so silently testing nothing.

So even if defaults are useful 9/10 times that 1/10 is so painful and expensive that it isn't worth it in my experience. The time spent responding to, debugging and fixing those outages far, far outweighed the time saved by the convenient default values in the 9/10 times.

defaults are a different thing to nil punning

So nil will have had special consideration in Clojure core functions

That doesn't mean it doesn't crash either it will absolutely be unhappy with nils in your math

nil is usually unexpected in test outputs or at the very least an unhappy path

In terms of debugging that's why I can't quit this language flowstorm let's me visually step through what happened line by line, backwards, forwards, programmatically - whatever

Languages should be competing against each other by their best time travel debugger it just completely removes the need for guesswork

I suspect there should be some data model guideline that says if keys can be missing then values can’t be nil, or if values can be nil then keys must be present. In the famous saying there are two hard things: naming, cache invalidation, off by one errors, I think one more to add is handling missing data.

For me: documentation at the "front door" of an interface, especially in that long moment before you decide to add a spec or Malli schema.

This is really the kind of thing you want to fail at compile time which isn’t real possible in a dynamic language like Clojure.

Unless you use Typed Clojure, which is a library. https://github.com/typedclojure/typedclojure

I haven't used it, so I don't know its tradeoffs; but its docs say its types exist at compile time: https://github.com/clojure/core.typed/wiki/User-Guide

Well it is possible - you can add a user macro that calls into clj-kondo (or anything actually) to check your codebase on compile

It just doesn't make much sense to do - most modern developers will be running static analysers through LSP or their editor (knowingly or not) continuously on code change so as to see those errors quicker than re-compiling the program

> ... if the key is missing today you'll get nil as the value and since Clojure is a nil punning language it usually does sensible behaviour in your program

And that is still doable AIUI: they're optional checked keys. The doc describing them makes the distinction between required and non-required keys.

Arguably we already had those: I religiously use spec'ed maps in my Clojure since a great many years (and Clojure spec is still in alpha, but "alpha" in Clojure land basically means: "more stable and less likely to change than any feature in any other language" and I'm only slightly exaggerating here).

In my case I use good old defn-spec (form Orchestra but YMMV) instead of defn. And my maps are (partially) spec'ed, using spec'ed keys (as well as any other non-spec'ed key I feel like using). Sure it's only runtime checks but it's really great.

You get to both have the extensibility (you can for example add keys that don't exist yet later on without changing any of your specs) and you can specify which keys are required.

For there is such a thing as maps where you know that this and that key must always be there.

I don't think it's an issue to have optional checked keys. Especially not when you can mix both required and non-required keys in the same map.

> if the key is missing today you'll get nil as the value

You can add a third parameter to override the nil if detecting the missing key matters.

(You almost surely know this, but not all HN commenters will.)

  user> (:bar {:foo 1})
  nil
  user> (:bar {:foo 1} :missing)
  :missing

Yeah I'm with you, this feels like a case for assertions & not a new core feature. Perhaps I'm missing something because it was promoted by Michael Fogus and has been ratified by those who would know (Alex I'm assuming). To me it doesn't pass the test of necessity as something needed at the core but at least it feels somewhat ideomatic.

For me it solves a real issue, a lot of the code that I work on looks like this:

  (defn foo [{:keys [a b c] :or {a 1}]
    {:pre [(some? b) (some? c]}
     .....

having `:keys!` will automatically remove the need for `:pre`

One thing to note that's maybe less obvious is that you can destructure some keys with the check and others without. This makes the function interface a bit self-documenting. At a glance you see that the username is required and other parts are maybe not.

    (defn my-function
      [{:keys! [username]
        :keys  [firstname
               lastname]}]
      (do-stuff username
                firstname
                lastname))

A minor downside is that now it seems `nil` is even more overloaded b/c you can explicitly pass in a nil and give it a special meaning. This generally cascades in to messyness (better to have a special key like `:missing-username`).

Feels like throwing an error on nil would have been better/simpler? But I'm sure there's an angle I've not considered

Some explanations from https://clojure.atlassian.net/browse/CLJ-2961:

> Clojure’s idiomatic use of maps has proven valuable, but missing required keys, misspelled keys, and invalid values can lead to failures that do not connect to the actual source of the problem (e.g. NPEs) making diagnosis difficult. At the same time, Clojure lacks a simple inline mechanism for functions to document and check the keys they require and accept. Existing tools either separate those expectations from the function itself or couple data shape and data provision.

This is actually great, and I predict that fans of nil-punning will rapidly discover the joys of actually having errors trigger where the error was introduced rather than propagating through the program.

Any news on ClojureScript gaining the feature?

Working on it :)

Amazing! What are the most interesting areas for ClojureScript in the future, if you don't mind me asking for some casual semi-serious prediction?

Thanks for everything you've done for Clojure and ClojureScript, I'd surely have dropped programming as a whole if I didn't discover Clojure and ClojureScript at the time I did.

Honestly what's mostly at the forefront of my mind is greatly improving the documentation around ClojureScript as well as our fork of Google Closure Library (GCL). At work we've switched to DataStar (a single JS include) and coupled that with ClojureScript/GCL - we no longer rely on anything from NPM, to call this a simplification would be a gross understatement. Bundle size is 30K gzipped and we spend no time thinking about our build or JS tooling/dependency tomfoolery.

So less about ClojureScript specifically, and more generally how I think we're well situated for people looking for a way out. The current mainstream practice dead end is bigger than the one that made React (also originally just a script tag include) appealing to me back in 2013. There are of course many ways forward that don't involve CLJS, but I think ClojureScript/GCL and the new crop of NPM-dep free pure CLJS solutions like Replicant are well situated for folks who can see that accepted practices are not delivering enough value even with AI assistance.

We switched to DataStar after seeing one of your vid about SSE. Full ack that calling it a "simplification would be a gross understatement".

Namaste!

Well, kind of. This is the kind of problem that you might throw schemas (eg. Malli) at pre 1.13.

It’ll be nice to have it at hand in the base language though.

What is nil-punning?

allowing functions to treat nil arguments as empty versions of their expected types

For example:

(cons some-value my-list)

If my-list is nil, then the above expression will result in a list containing only the element some-value. Otherwise it will be a list starting with some-value followed by any other values that have been previously in my-list.

Is it only me or this sounds a bit counter to clojure philosophy?

It's 100% opt-in at the call site and doesn't affect existing code, so no?

Many people (including myself) already have checked key variants for maps; this mainly extends the syntax to destructuring too.

The maps are still open to new keys even if some keys are checked. I think that fits in with how clojure.spec and Malli work already, but in a lighter syntax.

The maps haven't changed at all, this is a feature at destructuring sites.

Sorry if I wasn’t clear, I was referring to the maps that are being destructured.

As a Clojurist the standard pattern for ensuring keys-are-set before doing-something is not-as-elegant-as-this. Clojure is full of macros that do useful things :) Simplifying oft-used patterns into compact representations is very on-brand. Plus, you need this like, all the time.

This will eliminate two whole classes of errors: 1) where keys are supplied a value at an undesired nesting-level. 2) where keys are not-yet-set for some other reason.

For the many programmers who have to write in checks and verifications themselves for this, this saves quite a bit of time, removing the interruption from coding and restoring the flow of getting logic-to-symbol.

Seems additive to me; no breaking changes, and better control and error messages when opting in for it, seems entirely Clojurely to me.

Howso?

I agree it feels a bit counter to the philosophy of Clojure. It's adding new syntax to the language for map destructuring only (that will need to be implemented in cljs and other runtimes for consistency) and it's a purely runtime check as we don't know the map's keys at compile time. I don't see what new kind of safety it adds that's not achievable with existing solutions such as :pre or doing an assert inline.

I feel there are better solutions to this problem that already exist, such as using spec/malli and validating the value properly rather than just checking for presence.

One philosophy of Clojure is to facilitate building practical and robust systems. In practice, people do runtime checking of maps using punning, some kind of `nil` check, or a more ponderous `(get m k sentinel)` checking pattern. The new feature obviates the latter as destructuring syntax in the vast majority of cases where the absence of the key throws. It's opt-in, so if Malli/Spec work then you don't need to use this. I will say that we have some other things brewing that compose well with `:keys!` and friends and the 1.13 release is going to DRY a lot of existing and future Clojure code.

I love the idea of clojure and perfect immutability, but holy crap I cannot grok the syntax. My C-trained brain explodes.

It'll take a while but now other programming languages look alien to me. Once you've adopted s-expressions it is hard to go back.

It took about two weeks for me to be able to read it.

Might as well have been Russian.

Now it's as natural as any other language.

You get over it really quickly once you start actually using it. I find it basically impossible to read Clojure outside the editor in any meaningful sense.

It's like a light saber instead of a machine gun. Let the bullets come to you.

I went through a similar phase decades ago with Common Lisp. It takes a week or two. Now, it’s quite a natural syntax and I see the parens as a huge benefit. I like Clojure syntax even more than CL and Scheme because of the map and vector literals.

Lisp is tricky. Pretty much every programmer for whom it's not their very first PL hates it initially, but then there's a time, a threshold after which no other language feels more readable than Lisp.

Using structural editing idioms and the REPL, usually makes the process less vexing.

I think for some people it's an aesthetic knee-jerk reaction and the fact that it's "unusual" for a programming language. It's not like their brains "can't grok" -- it's just syntax after all. People don't tend to have that reaction to xml for some reason. It's data, so it gets a pass. But then this is code-as-data and they have to break that barrier conceptually and then get past the aesthetic knee-jerk reaction.

We just updated one of our projects to 1.12.5, but I might push for 1.13 as this could be very useful, although an alpha version might raise questions.

This is helpful, because practically many functions in the wild have assert-like checks at the top of the function, e.g.

`(if-not key1 (throw (Exception ...))`

...and pre-conditions, e.g. `:pre [condition1 condition2]` do not run when `assert` is off.

Ah yes, the missing seventeenth way to validate function parameters.

Slowly but surely dynamic programming proponents discover the value of statically verifiable correctness. Who'd have thought?

Snarkiness aside, this is hardly a static type check. This appears to be a runtime argument check, somewhat akin to the following python:

  def foo(*, a, b): return a+b

which errors out at runtime if `a` or `b` are omitted, despite being keyword arguments which are usually optional.

Oh we know. That's why static types are a la carte in Clojure. You can have them if you really really want them.

Reply on news.ycombinator.com