Tinholes – performant, strongly typed lenses in Clojure | Hacker News Reader

Tinholes – performant, strongly typed lenses in Clojure

blog.podsnap.com

60 pointsluu21 comments

This is exciting. Lenses are really powerful and ought to be used in more languages to accompany immutable data types. It'd be nice to see if this technique can generalize to traversals and prisms as well. In practice, I use those two in nearly any lensy computation.

Using macros to pre-compile the lenses is clever, but feels like a hack around typed-clojure instead of being aligned to it. All of the information needed to determine a lens' action is available at compile time even prior to expansion. Can a van Laarhoven representation be made in typed-clojure that recovers this information?

> type-safe, efficient, concise, idiomatic and flexible handling of deeply nested data structures, and that's what we got. Prima facie, these were unreasonable requests - it's not as though other languages were lining up to answer them

and since the line was crossed: yes, other languages are exactly lining up to answer all of those. You don't even have to burn-in the choice of lens. It's exciting to see this technology developing all around, but it'd also be nice to see it with a bit more humility.

> Using macros to pre-compile the lenses is clever, but feels like a hack around typed-clojure instead of being aligned to it.

I couldn't help but feel this way as I was reading it as well...forget compile time though. I wonder if there's any way/plans to eliminate this kind of type erasure period from typed Clojure eventually?

In Haskell you have appropriate typing for lenses because you build paths in a typed way. For instance, the API could like like this

    type Lens src tgt

    compose :: Lens src int -> Lens int tgt -> Lens src tgt

and then chains of lenses composed with `compose` would carry the appropriate typing information to not have to fall back on Any. This doesn't work "idiomatically" in Clojure because the *-in commands want to consume a list of keys where (1) each key is untyped and (2) the list cannot be heterogenous like it must be for the typed API.

That all said, it'd be easy to implement the typed API in Clojure as the "van Laarhoven" representation I mentioned has `Lens` just being a certain kind of function and `compose` just being regular (reversed) function composition: it's exactly like transducers in that regard (in fact, transducers can even be seen as a specific instance of a particular generalization of a lens, I believe).

The challenge is figuring out where the vtable for things like `map` lives. I'm pretty sure this is solvable in Clojure by using protocols (although I'm also pretty sure you can't get `Traverses` that way since you need return type polymorphism to get `pure` to work).

Anyway, it'd be neat to experiment. Maybe I'll teach myself some typed-clojure and see if I can't port the van Laarhoven lenses.

> since you need return type polymorphism to get `pure` to work).

If you don't mind an extra layer of interpretation (which you might very well mind!) you can get around this by building up basically an AST of your applicative computation---I did this, more or less, in my monads-in-clj library: https://github.com/bwo/monads which has a kind of not-that-useful applicative definition: https://github.com/bwo/monads

I don't know too much about the innards of van Laarhoven lenses but I'd guess that a straight port might be inhibited by the fact that Clojure isn't pervasively lazy---so using Const as your functor, for instance, won't let you avoid actually computing the eventual second argument which is just discarded. But I dunno.

I was thinking about the applicative AST. I think that's a good solution, and perhaps if you're willing to explicitly compile it then you'll only need to pay once (and maybe even memoize it on later uses?)

There shouldn't be a problem with strictness and van Laarhoven lenses. The particular trouble is just writing the lenses in the first place:

    (def one [inj]
      (fn [p] 
        (fmap (fn [f] (pair f (snd p))) 
              (inj (fst p)))))

where did the `fmap` come from?

I apologize for my miscommunication and ignorance. First, the bragging was supposed to be on behalf of the Clojure ecosystem rather than myself. Second, while I am aware of the standard lens implementations in Haskell and Scalaz, I didn't see that either dealt nicely with intra-path transformations, and the Scalaz version didn't seem very concise. If there are more examples, or if I'm wrong about these, I would love to learn more. I agree that the interaction with typed Clojure is not ideal, but at least there's an interaction of some sort, albeit possibly a "hack." Googling "van Laarhoven" has led me to a few hours of reading material and to SPJ's talk, so thanks for that...

Could you demonstrate a simple example of the intra-path transformation? I am not sure that I see exactly how it should be implemented, but so long as it isn't a lens law violation I think you can do it in Haskell's lens system at least.

I was thinking of setting ```:c``` in ```{:a {:b "{:c 5}"}}```. More generally, a leaf in the container tree that is not itself a container but can be converted to and from one. I need to play more with http://hackage.haskell.org/package/lens, but I'd certainly appreciate a shortcut to enlightenment...

So there are a few things going on here

    1. Focusing on "the subtree" instead of "the value
       at the subtree"
    2. Working with data types like `{:a {:b {:c 5}}}` is
       somewhat uncommon in Haskell (too little information
       gets projected into the types)
    3. Doing "along the way parsing"

Each of those are fine to do in Haskell and I could work up an example. Perhaps the best domain for this is using the lens-aeson[0] package since your data type is not so different from JSON. Here you can use the _JSON prism to do your "along the way parsing" (the failure mechanism of prisms catches failed parses naturally). You can also focus on whole subtrees naturally (since the JSON Value type is just a sum over subtrees and values all together, so the natural parametricity distinction is lost).

So I'd look at [0] for a wealth of examples in that vein. I'd also look into zipper lenses. They were removed in more recent versions (I think anticipating splitting them out into another package) but they're available in the 3.* series [1].

[0] http://hackage.haskell.org/package/lens-aeson [1] http://hackage.haskell.org/package/lens-3.10.1/docs/Control-...

Yes, lens-aeson looks pretty damn close. I've got a bunch of reading to do.

The existence of zipper lenses catches me by surprise, as I have been frequently directed at a stackoverflow [0] response on the difference between the two...

[0] http://stackoverflow.com/questions/22094971/what-are-the-dif...

I'm the author of the non-accepted answer to that question. For what it's worth, I think cgrand's is weak in describing both zippers and lenses, but ultimately lands on the wrong distinction.

Sorry about "performant," everyone. After years of being mocked as a pedant for avoiding the word (ok, there may have been other reasons), I decided to dip a toe in the descriptivist waters. Maybe it's too soon. I toyed with "fast" in the title, but that could be taken as referring to ease of use. I'll spend the next few hours trying to come up with a better title.

Ok, I managed to get a van Laarhoven representation in typed Clojure, though it wasn't exactly a smooth experience. http://blog.podsnap.com/vanhole.html

This seems to be the core of the idea, which I like a great deal:

"Type-checking is somewhat difficult with the current implementations of ph-whatever, as, intuitively, it would be with anything implemented as a recursive function that might consume and return values of different types at different levels of the stack. "

So the goal is to enable TypedClojure to work in some functions, such as assoc-in, that are recursive and difficult to type.

Disclaimer: I don't know Haskell.

But I did read the article - do I understand correctly that lenses are more or less equivalent to the Common Lisp getf constructs?

Performant?

Yes, performant. The article shows how a tinhole-based get-in is significantly faster than the default get-in.

(The methodology could be a bit nicer; it's a simple timed doseq, but it does show some performance gains.)

I'm not sure if you're asking whether this library performs well or are picking on the use of the made-up-for-engineer's-use word "performant." I'm going to assume the latter. I think it's an OK word to use. It's a succinct way of saying something performs well. We programmers tend to have a deep history in stealing terms from other industries and butchering their meaning (orthogonal, impedance mismatch), it's about time we make up some of our own!

Well, I have no problem with making up words, but I feel like mentioning that there is an even more concise term that could be used: "fast".

Although this word doesn't have a scientific gleam or a stamp of enterprise approval, it is actually more specific than "performant". Technical words are nice but sometimes they just serve to hide details. All that glitters is not gold.

Perhaps he didn't want the more specific word? Fast means fast, great, I get that. But what if you want fast code that works consistently, "performs" well, if you will. Then performant efficiently fills an appropriate niche.

I am not sure if that is the goal of the original statement, but the point stands nevertheless. All words were just made up at some point of time or another and it should be trivial to prove the set of words available in any language's vocabulary is a very small subset of the set of all ideas one could possibly express; for this reason I think the creation of new words should not be discouraged unless there is a compelling reason to do so.

In the end, the word accomplishes what language ought to: it conveyed an idea which was more or less understood by the target audience it was aimed out.

* it is actually more specific than "performant"*

I disagree. See this use of 'fast' in a submission, for example: https://news.ycombinator.com/item?id=7132044

What does "fast" mean there? It could mean the development environments are performant. Or it could mean they're quick to implement or set up. I believe the latter is what is intended, but I can't be sure from the submission or even the article, because the word "fast" is not very specific. In this context, the word "performant" would have been much more clear.

Language is fuzzy and malleable and subjective. People shouldn't be so quick to lay down declarations of approved or preferred words. And to chide someone for using a word that hasn't made it to the dictionary yet is tantamount to declaring that we should freeze English and never change it (referring to the great-grandparent post here).

Reply on news.ycombinator.com