Functional-navigational programming in Clojure with Specter | Hacker News Reader

Functional-navigational programming in Clojure with Specter

nathanmarz.com

105 pointsdantiberian33 comments

Wow, I love this. My impression is that Specter is (at least in main use cases) Haskell's lens package. But while lens is very inside-baseball (at least in the tutorials I read), this explanation describes an extremely useful principle.

Every language needs a lens/Specter. If you're spending a lot of time messing with data, this article will go directly to your heart.

Exactly what I thought, either lens or zippers.

Definitely more lens than zipper. If you think of a lens, in simple form, as a path into a structure then a zipper is a structure plus a path into itself. This constrains the form that a zipper can take.

If you examine these paths on their own then you get into general "optics" which is where the lens type lives and where specter lives (it seems). In particular, we start talking about generalized folds and traversals.

I think https://www.fpcomplete.com/school/to-infinity-and-beyond/pic... does a nice job of explaining a lens:

> At its simplest, a lens is a value representing maps between a complex type and one of its constituents. This map works both ways—we can get or "access" the constituent and set or "mutate" it. For this reason, you can think of lenses as Haskell's "getters" and "setters", but we shall see that they are far more powerful.

I've read about specter before on its github[1], and I must admit I didn't see the bigger picture that is explained in the opening example of this article.

I thought I could get by with just using:

    (-> huge-thing :key1 :key2 first ...)

But now I actually grok the value behind Specter! Great stuff, and I will use it next time I begin up a re-frame[2] app!

[1] https://github.com/nathanmarz/specter

[2] https://github.com/Day8/re-frame/

Presentation from StrangeLoop 2015

"Specter: overcome your fear of nested Clojure data" https://www.youtube.com/watch?v=mXZxkpX5nt8

> Any connection to transducers?

Yep!

On the fancy Haskell lens hierarchy (hackage.haskell.org/package/lens) we have the notion of a fold or a "getter which touches multiple items". The existence of a fold for a type like `Fold s a` indicates that we can extract from the type `s` some number (0 to many) `a` values in sequence. This is the idea of "Foldable" in Haskell.

Given a foldable type `s` and a transducer we execute the transducer by passing the "build" reducer in and then "visiting" each value `a` inside of `s` with the reducer that the transducer returns (modulo the early stopping bit which is just sort of a Clojure-specific optimization). Essentially, the transducer is a notion of "visitation" which is invariant to how the final summary is constructed—essentially the same thing that's captured in the "getter which touches multiple items" of a Fold.

So there really ought to be a way to treat any specter optic as a possibly very limited transducer. Essentially, the "read" component of a lens will correspond pretty directly.

We can also see this by remembering that any pure transducer is semantically equivalent to a function `a -> [b]` which you can read as a way of finding 0-to-many `b` values "inside" of `a`.

I like Specter, but it doesn't quite give me the holy grail it promises. My main pain-point is that it assumes I know the route to the datapoint I want to change. What if all I know is the general shape of a subset of data, and also that that shape might appear none or more times in my client dataset?

I'd love to be able to describe features of a data structure and have all qualifying elements be transformed, without my having to tell the select function how to get to those data structures in the first place; just hand it a data structure, and have it grovel through looking for matching subsets, then transforming them as desired.

Sort of like a grammar/DSL for Clojure data structures, without having to be specified from the basic character level up.

It sounds like you're looking for the "walk" selector, which comes with Specter:

(select (walker number?) {2 [1 2 [6 7]] :a 4 :c {:a 1 :d [2 nil]}})

=> [2 1 2 1 2 6 7 4]

I've personally found this to be a use case that comes up infrequently.

Walk shows up quite often in the "uniplate" functionality of Haskell's `lens`. It's really, really useful for handling AST transformations, but depends a bit on laziness here.

I'm doing a lot of tree transformation stuff at the moment where it comes up a fair bit. Thanks for the pointer, I'll give it a try.

I like the idea.

Here's another that seems similar to it: an immutable language wherein you could create a mutable, write-only copy of an object. After writing all that was required, the copy could be finalized, at which point it becomes read-only (hence immutable). This is a bit like the builder pattern, but I've never seen a language with built-in support for it.

Thoughts? Is it much different than what the OP proposes?

Clojure/script already has this. Its called transients

Also check out traversy, by the super-talented Chris Ford. Even works in Clojurescript. :)

https://github.com/ctford/traversy/blob/master/README.md

Specter also works with Clojurescript :)

This is pretty nifty. What is so exciting about Clojure is that the language does fully realize the original lisp intention of creating building blocks on which any paradigm or style of programming can be built.

How exactly was that not realisable before?

Common Lisp is a multi-paradigm programming language, arguably more so than Clojure because it has a fully featured multi dispatch object system as well.

Oh! I'm glad to hear about this. I was getting painfully close to having to implement it myself...

Does something like this exist for JS?

The other name for this kind of abstraction is a 'lens' (as touched on at the bottom) - I found this library pretty quickly: https://github.com/DrBoolean/lenses

I find the use of all-caps to be non-idiomatic. Take `LAST`, for example.

Regular functions are treated as filters, so that's done so that things like ALL, LAST, etc. are clearly delineated as something different than filters.

It is stylistic and idiomatic preference. I'd probably group those kinds of functions into a namespace. Then again, I didn't write the library. :)

I don't understand how it's doing it, but it looks seriously cool - like a paradigm I want to start programming with.

To whoever downvoted this - instead of doign so, why not point me to a beginners-guide? I have read whatever I've found so far...

We definitely need an elixir port.

José Valim talks about current details and future plans for Elixir 1.3 here https://groups.google.com/forum/m/#!msg/elixir-lang-talk/DHF...

Exactly what i need! Thanks!

Reminds me of SPARQL.

Reminds me of XSLT.

Lenses (as a construct) can encompass traversal, getting, setting, and transformation. With this in mind, I'd suggest that lenses are closer to a combination of XPath and XSLT. A key difference is that no schema is imposed.

That said, I'm a little sad to hear that lenses remind you of an XML technology. :) I suppose I'd rather that XSLT be perceived as a very particular way of doing data transformation under particular conditions with a particular syntax.

All of the above said, I think good lessons could (in theory) be learned from XML, but I feel like XML's notational particulars get over-accentuated and the underlying thinking about data structures gets muddled and even lost.

Good points.

I just realized that when I think of 'XSLT' I tend to think of it in terms of the combination of XSLT and XPath.

Which is exceedingly reasonble to think, since XSLT actually mandates XPath expressions for all select attributes. However

The major conceptual difference I can see is the ability for (apparent) in-place editing/updating, the lack of which was/is a major pain point in XSLT.

As XSLT is a functional language for traversing and manipulating heterogenous trees, one can expect quite a lot similarities beneath the surface syntax for most solutions targeting the same problem. At least in part because the limitations of the users (us) and the problem to be solved tends to constrain the solution space for generic (library like) solutions rather strongly.

Reply on news.ycombinator.com