Clojure on Fennel Part One: Persistent Data Structures | Hacker News Reader

Clojure on Fennel Part One: Persistent Data Structures

andreyor.st

165 pointsroxolotl21 comments

Clojure's immutable HAMTs are still a superpower nearly 20 years later. They've been copied in pretty much every language as a library (I did so myself in Zig) but what really makes it work well in Clojure is the fact that they're built into the language, so the entire ecosystem is built around them. Libraries that were made independently usually fit together like a glove because they are all just maps/vectors in -> maps/vectors out.

I thought it cheesy at the time. Then I tried clojure.

"The value of values". Indeed. Q.e.d. No notes.

Lisp programmers know the value of everything and the cost of nothing.

Apart from Andy Gavin [1]. He knows both the cost and the value of everything.

[1] https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp

sbcl's optimizers knowing the cost of everything is the gamble, no?

Yeah, they are so underappreciated - the practical differences in designing, delivering and maintaining software are real. Initially you see small differences "What's the point? I can write that in Python too... maybe it's not as delightful, but who cares?...", etc. Yet over time small annoyances accumulate and become an endless stream of headaches. I see it over and over again - I work on a team where we have codebases in different languages, and some services written in Clojure. Immutability by default is a game of a different league.

Yeah. I do wish there was something that was like Clojure with a TypeScript or Go-like nominal typing, but I do feel myself missing types a lot less with Clojure compared to other languages.

Type annotations mix poorly with s-expressions imo. Try an ML, which answers the same question of "How do we represent the lambda calculus as a programming language?"

There's already type annotations in Clojure and they look fine (though are a bit noisy). There are type algorithms that don't need annotations to provide strong static guarantees anyways, which is the important part (though I'm not sure you can do that with nominal types?). I think TypeScript and Go's syntaxes are a bad fit for s-expr but the idea probably isn't.

Isn't this like saying types mix poorly with ASTs?

HAMT -> Hashed Array Mapped Trie for those wondering.

https://en.wikipedia.org/wiki/Hash_array_mapped_trie

(The acronym is expanded in the article, but a ways down.)

I think I know and love more about clojure than any language I actually use.

I rewatch hickey talks anytime Im a little stuck on a big problem/idea and there's almost always another "ah ha" moment for me.

If the places i work were already on the JVM, i would have switched a decade ago, but I've been in .net world my whole career.

Have you thought about moving to a Clojure shop?

You don't need team's approval of your choices for using a tool, a technique, an idea. A tool remains useful even if nobody else but only you understand its practicality. There are days when I deal with projects where some don't even know about Clojure's existence, but I still may use babashka to understand the data flow. Having a Clojure REPL around is immensely handy, slashing through data that flows down the APIs is the best - JSON is so lame and annoying to deal with - even LLMs often can't get the jq filter syntax right, or when they do, it looks terrifyingly cryptic and confusing. Or sometimes I'd fire up an nbb REPL with Playwright - just to reproduce some issue without having to manually click through things.

Didnt they move to CHAMP? Otherwise, that seems like a waste of resources. They are literally the same but CHAMPs are a little faster in just about every way.

I am still a bit disappointed that they didn't change to RRB trees or copy the scala vectors instead of the built in ones. Iirc the scala vectors are faster in general while providing a bunch of benefits (at the cost of code complexity though, but even a better RRB list implementation instead of the scala finger trees would allow for that).

I wrote an RRB tree implementation in c# just for fun [0], and while they are harder than the tries of clojure, the complexity is pretty well contained to some functions.

0: https://github.com/bjoli/RrbList/tree/main/src/Collections

> Libraries that were made independently usually fit together like a glove because they are all just maps/vectors in -> maps/vectors out.

This is also the biggest weakness of Clojure (IMO). When everything is "just data", you spend a lot of time digging deep in libraries trying to figure out exactly what shape the data should take. Additionally, the shape of input data is almost never validated, so you spend lots of time debugging nasty type errors far from the place where bad data entered the program.

There have been some abortive attempts at solving this with things like spec, Prismatic Schema, etc, but nothing that has taken hold like TypeScript did with JS.

I'm still waiting for my dream language with the flexibility and immutability of Clojure, but without the pain points of an anything-goes attitude towards typing and data shape.

> When everything is "just data", you spend a lot of time digging deep in libraries trying to figure out exactly what shape the data should take

or that is a good spend of time where you familiarize yourself with said library (as they say, the documentation is the code!).

Usually the library is well written enough that you can browse through the source code and immediately see the pattern(s) or keys. The additional experimentation with the REPL means you can just play around and visually see the data in the repl.

A spec does similar (and it does make it easier to seek through the source to find it).

I've often got value from digging through libraries (in other languages), but I've almost always had the feeling that I'm not "doing it right", or that someone somewhere isn't "doing it right". Logically, the concept of encapsulation doesn't extend to meta-coding, but it feels like it should, by symmetry. It feels like I'm breaking encapsulation if I use the knowledge I gain from poking around when I code to the library.

I'm fumbling at the concept of a library surface not being self-describing but I suspect I lack some concepts; does this thought lead anywhere? Can anyone give me a clue?

I suppose if you examine the behaviour of a library, and code against that, then it is possible that the behaviour is unintentional and thus you end up being locked into a bug. This is most clear when the library is supposed to follow a standard (e.g., parsing some format), but is bugged and didn't do it right - and you code against that buggy behaviour.

However, that's an extreme case imho - you do that when you can't fix that library's bug or wrong behaviour.

But for things like key names and such, i dont think this applies - those key names are part of the library's api - and i often find that clojure libraries don't document them (or do but it's one of those auto-generated docs that dont mean anything).

> When everything is "just data", you spend a lot of time ... trying to figure out ... shape the data ... > you spend lots of time debugging nasty type errors far from the place where bad data entered the program.

I've been using Clojure for over a decade in various domains, different projects, diverse teams, etc., and quite honestly I don't even remember the last time it felt to me that way. Briefly, for a few weeks in the beginning perhaps it did. But at some point, maybe the REPL-driven workflow model internalized or something - I just don't ever feel like the way you describe. You're looking at the data as you build, so "far from where bad data entered" rarely happens - you watched it enter.

If anything, Clojure has spoiled me - I get annoyed having to dig through confusing type/data mismatch in some other languages, despite their sophisticated type systems. Uniform data structures mean your mental model transfers everywhere. You're not learning 50 bespoke APIs, you're learning map, filter, get, assoc. The real question is what failure modes you'd rather have. Clojure's tend to be runtime, but local and observable. Some type systems trade that for compile-time errors that are... also confusing, just differently. At the end of the day, sorry for a cliché, but it is a "skill issue". I can endlessly complain about my confusion with type systems, but a seasoned Haskell developer doesn't feel lost in types, just like I don't feel lost in Clojure without explicit type annotations.

> .. abortive attempts at solving this ...

I'm curious what you found inadequate with the existing solutions?

I remember when I started writing Clojure I'd use stuff like Records to encode the data-shape. Paired with Protocols they're quite powerful when the interface is more important than the shape. But in most situations the flexible data shapes are what make programs very easy to extend/evolve.

Reply on news.ycombinator.com