Clerk: Local-First Notebooks for Clojure | Hacker News Reader

Clerk: Local-First Notebooks for Clojure

github.com

130 pointstosh29 comments

Hey! We're excited to finally open source Clerk. Clerk is a local-first notebook, built on top of regular Clojure namespaces.

It can take a namespace like https://github.com/nextjournal/clerk/blob/fd79f7504ebffaa279... and turn it into https://storage.googleapis.com/nextjournal-snapshots/clerk/b...

The main idea behind Clerk is letting folks use the editors they know & love and not force them to edit code inside a web browser. (This came to us only after working for a few years on nextjournal.com where folks can edit notebooks in a web browser but better late than never :)

In order to keep the feedback loop fast Clerk keeps a dependency graph of all forms and vars in a notebook to figure out what changed between runs on a notebook and only runs those.

There's no homepage for it yet but Clerk is running a handful of sample notebooks on every commit and building a static website out of them. The latest version is at https://storage.googleapis.com/nextjournal-snapshots/clerk/b...

> In order to keep the feedback loop fast Clerk keeps a dependency graph of all forms and vars in a notebook to figure out what changed between runs on a notebook and only runs those.

This sounds interesting. Is it essentially an abstraction where to the programmer it seems like the whole notebook runs again for every change? Almost similar to how React makes it seem to the programmer that they are regenerating the entire page?

That's exactly it, yeah. It looks like a hot reload on save, but it does only the minimum calculation required.

From the README

  Specifically Clerk wants to address the following problems:

  - Less helpful than my editor

  - Notebook code being hard to reuse

  - Reproduction problems coming from out-of-order execution

  - Problems with archival and putting notebooks in source control

I encounter these problems every time I interact with a Juypter notebook and think about a Clojure style notebook environment :)

Great job!!!

What do most here use Clojure for? For me, there was little value in using it over nodejs for single server CRUD apps with postgres. Some things I liked are namespace keywords, REPl integrated with your editor, immutability. But still, it felt like swimming against the current. Immutability is not as useful in a single-threaded event-loop like nodejs. No third party SDKs, even though I know Java using the Java SDKs was tedious labor, most Clojure wrappers are pretty bad too, so more interop.

You can use Clojure on Nodejs too.

I personally can’t use it for production code. But whenever I automate something or write some prototype I use Clojure. Iterations are faster, more direct, less of a hassle.

There has also been someone commenting around here that they use it as an network repl to poke around running production systems, which I find an interesting use case.

I've been a Clojure hobbyist for many years, and these are a few of the things I've done repeatedly:

- Scrape webpages with http-kit and Specter

- Visualizations with Quil

- Serve static pages with Ring, Compojure, and Hiccup

(un)Fortunately, I don't write Clojure professionally, so I can get away with liking it for aesthetic reasons as much as anything. It definitely does feel like swimming against the current sometimes, though - one runs into bus factor issues a lot in the Clojure ecosystem. But hell, I love it all the same :)

Personally, I am really excited for Clerk - this looks like a really useful tool and I'm looking forward to trying it out!

Got any links to stuff you’ve done with Quil?

Yeah, here's one I put together a few years back: http://pontingdynamics.org/tri

> What do most here use Clojure for?

That's to evade having to use Java, Scala or Kotlin.

Clojurescript: because it's the easiest (compared to Elm, Purescript and Rescript) to call JS.

> Immutability is not as useful in a single-threaded event-loop like nodejs.

Concurrency is not the only, or even primary, benefit of immutability in software. State and state management is the number one driver of complexity in software, and complexity itself is the number one problem that makes building software hard. [0]

> For me, there was little value in using it over nodejs for single server CRUD apps with postgres.

You can use clojure on nodejs with the value that clojure is a much, much better language than javascript.

[0] http://curtclifton.net/papers/MoseleyMarks06a.pdf

I was referring to Clojure's immutable data structures, not immutability in general. My nodejs apps are just transformations pipelines of pure functions with postgres being my global mutable state. Clojure doesn't offer me anything here that I can't easily do with JS/node.

Sure Clojure is a better language but not better enough to justify the additional abstraction layer IMO.

If you're using pure functions then you'd likely profit from clojure's data structures, as they're much cheaper to copy and modify than javascript's native structures. Of course you could use something like ImmutableJS or Immer, but then you're putting yourself at odds with the rest of the JS ecosystem and built-in functions and you have to transform values back and forth. This gets a lot easier with Clojure(script) (even though there's obviously still some casting back and forth when you're going outside the CLJs ecosystem)

> Sure Clojure is a better language but not better enough to justify the additional abstraction layer IMO

Speaks for yourself, I'd take a pay cut to be able to use Clojure over JS (don't tell anyone )

Some of the why that is is captured here: https://gist.github.com/didibus/6e5ff960b25fdc1a5f8f97acede6...

But understand that a lot of it is about what language you personally enjoy and prefer, Clojure makes programming 10x more fun for me, and I already am someone who finds programming in general fun.

I sorely miss Clojure's immutable data structures when I use other languages (Python, Javascript, C++). They give me a guarantee that nobody else is going to be changing the data, somewhere deep down the call stack. I've had cases in production Python code where some seemingly unrelated code module somewhere was mutating data without my knowledge, breaking whatever I was working on. This is much harder to have happen in Clojure (you have to go out of your way to store things in mutable types, or use Java interop hacks to do this, which shouldn't pass code review without good reason).

Clojure's sequence abstraction also leads to a very rich and powerful set of core functions that make transforming your data a breeze. Other languages can do this too, but in Clojure I find its just really easy and natural since its part of the language's core library.

It also helps emphasize pure functions: data in, data out. Which makes my code much nicer.

Besides that, I use clojure because I like reitit and spec/malli for writing my HTTP routes, and hugsql for writing my SQL, both much better than the equivalent libraries in other languages I use.

I also like its emphasis on data first, code second. Sure you can do this in other languages too, but Clojure encourages it since its something the community has rallied behind.

They give me a guarantee that nobody else is going to be changing the data, somewhere deep down the call stack. I've had cases in production Python code where some seemingly unrelated code module somewhere was mutating data without my knowledge, breaking whatever I was working on.

Since we are talking about single-threaded single stack context, immutability wouldn't have saved you here, even with immutability you could have receive the wrong value/data from some function in another module down the call stack. The problem was not mutation but a function returning bad output.

You are implying some code modified data underneath you, which can't happen in a single thread.

An aspect of immutability I really appreciate is that I know no variable has been changed after it's been assigned to, including by me. If I want to know where the value was assigned, I only have to look for the only place it could have been assigned. I don't have to search anywhere else.

99% of variables are local to your function, is not like program a thousand of global variables.

Sounds like you code pretty tight. I've seen plenty of functions reference variables out of function scope, or be reassigned within long function bodies. I personally don't code that way, but not everyone does. And as you've seen in this thread, there are people that do like immutability. To each his own.

I do prefer immutability too and think it should be the default for most programming, my original point was that for me, the trade-off of using clojure (small ecosystem) instead javascript for single server CRUD apps is not worth it but as a language I'll take clojure over javascript any day.

This was in a single-threaded application. Immutability would have saved me because it guarantees that no nested function call can mutate data unless its passed in or stored in a mutable way (in a Clojure atom, for example). If the data structure does not contain any mutable constructs, then it cannot be mutated in a way that is visible.

> You are implying some code modified data underneath you, which can't happen in a single thread.

Sure it can. I mean, yes, the code is ultimately called by my code, through some deep transitive dependency, but the point is that if you have a complex architecture where dependencies interact in complex ways, things can be mutated by a nested function somewhere deep in the call stack. Immutable data would prevent this, unless the data is stored in a mutable object, but in that case, its easy to audit your mutable objects to see where they are used, accessed and mutated. In a language without immutable types, any assignment operator could potentially be mutating something that you don't want it to mutate.

Besides, many otherwise-single-threaded applications we develop these days do have ways that things can mutate asynchronously: request handlers, timeouts, animation frames, network response callbacks etc. It doesn't literally have to mutate them in parallel with other code I'm executing for it to become a problem, just when you flatten a particular execution chain the data may have changed without your knowledge between two steps. For example: I do a network request to an API and wait for the response, has the data been mutated when the response callback is run?

It may even be perfectly valid to do so, my main concern is that it should be explicit and carefully managed. If data defaults to immutable and mutable data requires some extra ceremony (ie deliberate intent by choosing to use an atom or other mutable type), then this signals where care must be taken and makes it clear where integrity may need to be checked, data may need to be reloaded or recalculated etc. Everything else is guaranteed to have the same value you left it with, no matter what, even if it was passed by reference to a callback or deeply nested function call.

Anyway, I was just pointing out why I miss Clojure's data structures when I use other languages and in what cases Clojure would have saved me from headaches. YMMV and all that.

I use it for everything.

Professionally, I write backend services (mix of micro-services and bigger SOA ones) of various types. Some command lines, some scripts, a few web apps.

In my spare time I use it for random things, mostly stuff I mess on, personal scripts, doing programming challenges, websites, data exploration, etc.

You can use Clojure with NodeJS as well using ClojureScript by the way, or NBB if you want something more straightforward for writing NodeJS scripts.

It's a good default app dev language for web backend and frontend projects at work, and attracts good programmers to the team(s).

Immutability benefits are first and foremost about semantics in most cases.

How crazy would it be to take something like this, compile it with ClojureScript and embed it in something like TiddlyWiki? I could see something like that being a nice way to hyperlink the more static documentation to the calculated results.

Sorry for the tangential question - does Clojure play nice with org-babel? Can this notebook interface be used with org-babel?

Clojure certainly works with org-babel, though in my experience, there must be done one thing - You have to start Clojure repl, and then jack in with cider, and only then you can evaluate code blocks.

Here is a bit more info regarding it itself. https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-...

Babashka works exceedingly well with org-babel, much nicer than using Clojure directly as you don't have to worry about setting up a project, namespace and connecting with cider.

See the following gist: https://gist.github.com/adam-james-v/f4d2b75a70b095d14a351a1...

And example code: https://github.com/adam-james-v/bb-draw/blob/main/bb-draw.or...

Look, a Lisp Machine. :)

Jokes aside, good work.

Gfgeftela?

Reply on news.ycombinator.com