From C++ to Clojure: Jank language promises best of both
thenewstack.io
thenewstack.io
If I could make one tiny plea, it would be to focus on tooling too, and ensuring the experience for someone trying Jank out is as smooth as possible. Don't assume everyone is already set up with paredit and can fire off emacs chords without a thought. I suspect that Jank will be of particular interest to C++ programmers, many of whom are used to a very different dev paradigm.
The Clojure community has done a great job at trying to smooth out the rough edges of Lisp tooling, and ensure there are on-ramps for newcomers (e.g. things like Calva for VS Code). I hope Jank keeps this up, because those first impressions really do matter. I'd hate to watch people bounce off Jank because they get stuck on trying to figure emacs out, or because they get frustrated trying to keep parentheses matched in Notepad.
Clojure - like many languages - seems to be benefiting a lot from the work the IDE people have been putting in to good infrastructure. A lot of the magic is being moved out of Emacs towards tools like Treesitter and the LSP server which are more platform independent. Not a huge amount to do with Clojure per say, but the Clojure ecosystem is benefiting a lot from it. The Emacs specific stuff is becoming very Emacs-specific (like paraedit, bless its socks).
I aim to focus a good deal of time, post jank's launch, to create materials which enable this. Materials specifically targeted at OOP devs coming from the native space.
Maybe I should play some advent of code games with it to get comfy!
For example, I’ve always thought that the interesting part of Erlang is Beam & the uninteresting part was a niche language. Beam with JS as the language is much more approachable & leverages a rich existing ecosystem of tooling & libraries. Indeed, if you look at it the right way Cloudflare Workers is exactly that; isolated virtual processes within a single OS process that communicate with each other using a message based IPC & if they crash just the microservice is taken down & transparently relaunched.
So from that perspective, why Clojure instead of the same features on a different more popular language?
A lot of responsibilities move to the edges and you're left being able to think and tests pieces of code in isolation. The repl becomes usable, and you code from inside your running program. It's a steep step to get started but it's going to transform how you think about some problems.
Also: no one has managed to explain to me what makes emacs so uniquely capable of interacting with a running interpreter, despite this usually being the USP I hear for “needing” emacs when attempting to Lisp.
Since I’ve got you here, can I ask you for your thoughts on LispE and how it contrasts with jank’s approach? It seems clear that they both arrive from different lineages of the Lisp kingdom. LispE is also found dangling a few toes at depth within the icy waters of array programming.
Perhaps the only connection between the two is that the one makes me think of the other. But if there’s more, though, I’d love to read about it.
Different lineages, for sure. Clojure really stands on its own in the lisp world and some die-hard lispers claim that it's not a lisp at all. However, for Clojure devs, I think we generally aren't interested in using the other lisps in practice, for building practical software. We just appreciate them for their lispiness. So the main difference will be that LispE is more like CL than like Clojure.
Aside from that, LispE is interpreted, whereas jank is JIT compiled with full AOT compilation support, using LLVM. By using Clang/LLVM, jank also has full access to C++ interop, whereas most interpreted lisps are sandboxes on their own.
I'm not familiar with this side of traditional lisps very much. Someone may be able to jump in to embellish or correct.
The javascript/UI people have found live reloading/editing to be a game changer, but this has been the case for lisp development since...well, the beginning!
The 90s with C++ and Java broke with history. Thankfully the rise of web apps has given us iteration speed back!
Visual Age for C++ inherited the Smalltalk experience, alongside Energize C++,provided a similar experience, but were too expensive for early 1990's hardware and too resource hungry.
Live++ brought the experience back to game developers.
Java has supported partial live reloading since early days, and for those willing to pay for it, JRebel takes the experience further, back to Lisp/Smalltalk.
Emacs is easily programmable (and in lisp), more unixy than modern Linux, like a whole operating system. Lispers make tools for it because it's very easy for them. Lisp tools have been far advanced of others; LSP wasn't initially adopted in Lisp communities because better tools (swank) were already around.
LSP and Treesitter were embraced if not shortly, at least eventually. Lispers are well-known, almost notoriously, for "stealing" good ideas from other places - Clojure's transducers, core.async, STM, refs and atoms, core.logic; Common Lisp's condition system, multi-dispatch in CLOS, SBCL's compiler notes, SLIME's debugger - ideas for all these things originated somewhere else.
Does vim display inline graphics in a repl, with editing capabilities?
This is one of the basic features of classical graphical Lisp environments from the 1980's, IPython and Jupyter notebooks, or better, Mathematica are only building upon this.
Emacs and vim are not quite at the same experience level.
As an Emacs user that has to work in teams of non-Emacs users, the answer to this boils down to culture and ecosystem maturity.
Purely technically, there's nothing that Emacs allows me to do with REPL Driven Development that _couldn't _ be done in another editor. Practically? I still haven't been able to even get (sufficiently) started with any editor.
I often have to work on a Rails codebase that takes tens of seconds just to start. Dev cycles in the traditional Edit-Restart flow were so painful that I wished badly to burn this codebase down and rewrite it all in a language and style that supports a Lisp-style REPL. Then I discovered inf-ruby.el in Emacs. It allows me to just edit the code and reload only what's changed. It automatically inserts the correct `module Xyz; ...; end` etc. No more restarts.
I showed it to my coworkers. They shared the pain with restarts. And yet, to this day, none of them have an equivalent to inf-ruby in their editors.
inf-ruby.el is less than 1400 lines of code. It's easy enough to implement in Vim, IntelliJ, VS Code, anything really. But it exists only in Emacs, so far. Why? I'm sure because some Emacs user, once upon a time, wished similarly to have a more Lispy REPL for their Ruby dev work, and just built it. Because they were used to it, from having used Emacs or other Lisps with Emacs.
Compare that to when I first tried to support Common Lisp development on VS Code. A language that already has full support for REPL Driven Development with an interactive debugger built-in. Nope, we aren't gonna use any of that, because in VS Code land, we use LSP. A model that really only works for languages that can be statically analyzed. You want Go To Definition on a method that's included from a module and called on a receiver that's a dynamic variable? Well, sucks to be you, I guess.
So, while people make do with the severe limitations of Solargraph or Ruby-LSP, I use robe.el — which adds lisp style code intelligence by running inside your Ruby process — and get code intelligence that actually works with a language as dynamic as Ruby. Again, robe.el was just there for me to use. Again, there's nothing about Emacs that makes robe.el possible in Emacs but not in other editors. Again, there's no equivalent to it (that I've found) in the ecosystems of other editors.
I wonder if it is an exposure issue: only those who use emacs know it is possible and those who know emacs have little incentive to leave their editor-OS.
I guess I’ll have to dig into some of these examples you have shared and see if I can comprehend and/or relocate their magic.
Emacs is weird; some people "get it," some, even after using it for years, just never do. The thing "to get" about Emacs is a knack for quickly automating things. Those with shallow exposure to Emacs think that Emacs users are doomed to tinker with their configs all the time. In reality, that's not entirely accurate. I can share so many fascinating, practical examples where I needed to get something done on my computer, and Emacs either already had all the pieces required or provided me with facilities to build upon.
Exhibit A: One day, while taking notes and having to jump between multiple web pages in my browser, I got irritated by having to jump to the browser, finding the tab, going back to my editor, etc. I wrote a function that lets me control my browser tabs directly from Emacs. Why? Because it's convenient, and because it wasn't that hard to make.
Exhibit B: Few days ago, my colleague was showing me something over Zoom. I didn't want to derail his train of thought, I didn't want to keep interrupting him with: "hey, wait, don't scroll away just yet", "can you share that link with me?", "whoa, hold on a second, I need to write that down", etc. Over the lunch break I decided to solve this problem for myself. I use Flameshot. I checked if there are any plugins for it. Turns out there's an open GH Issue, that's all. So, I wrote a command that checks ~/Desktop folder - that's where my screenshots get dropped, then finds the last .png (if it's created less than 2 mins ago, otherwise prompts for a file), sends it to tesseract (a tool, the existence of which I haven't heard until that moment), then opens the OCRed text in a buffer. Now I can quickly select any area on my screen and retrieve text from it with a single keystroke.
Exhibit C: I use Google Translate directly from Emacs. Which by itself is nothing out of the ordinary. Pretty much every other editor has some kind of plugin for that shit. I was reading articles in a foreign language I'm learning, sending pieces to get translated - again, nothing new here. However, it doesn't translate numbers, and that's normal and totally expected. Yet, I wanted to see numbers in their written form. What did I do? I found the function it calls before sending text to GTranslate API, and using advising mechanism, wrote a function that right before sending a request, searches for numbers in the original text and turns them into written form, and then sends that for translation. Advising feature is extremely powerful and it doesn't exist in any other editor beside maybe Lem - neither VSCode, nor Vim, nor IntelliJ, nor Sublime has that stuff. Did I have to look up GTranslate API docs? Nope. Did I really need to sift through the Emacs google-translate package code? Not really. I just needed to find the function responsible and needed to know its signature. Took me less than 15 minutes and fewer than 30 lines of code, most of which is my comments explaining the hack. I couldn't even find a generic Elisp function to translate numbers to words, I just used some npm package for that.
I can tell you many stories like that: the million reasons why I can't ever abandon Emacs and why I love it. I don't care that traditionalists using IDEs think I'm delusional, I've seen that world - it has its perks but also limitations. My world of Emacs not without its own drawbacks yet it allows me to hack some stupid shit almost effortlessly. I guess tis ain't that stupid if thy shit actually works, eh?
It's not a "prerequisite" per-se, although I guess you just never stumbled upon the fundamental truth about Emacs. In fact, it's not just an ordinary editor, it's a "Lisp Machine". Of course Emacs is far from the actual implementation of a Lisp machine, therefore the quotes.
Emacs feels like a Lisp REPL with a built-in editor, and not the other way around. Once you immerse yourself into the world of Lisp, sooner or later you will realize - Emacs is far from the ideal implementation of a Lisp machine, yet it's the best practical one we have today.
Chords, shmords, modes, modality, non-modality, keymaps, keybindings, etc. - that all is just "higher-level interface" you can change to your liking. I consider myself a die-hard Vimmer, never ever thought about switching to Emacs, until one day I woke up with realization that Emacs actually vims better than Vim and Neovim.
There's something "magical" about Lisp - coders either love it or hate it with passion, but anyone who accepts it, with all its flaws and powerful features, those who embrace the dynamism, the malleability of the structure - they may open a beautiful world of countless fascinating ideas. And once someone opens their heart to Lisp, either with sadness or with joy they sooner or later realize - Emacs is just the best practical tool to work with Lisp, there's simply nothing better. Even though "the best" doesn't mean "the best possible", it just means "the best today".
I quit my job to work on my programming language - https://news.ycombinator.com/item?id=42658898 - Jan 2025 (32 comments)
Jank: Programming Language - https://news.ycombinator.com/item?id=42477992 - Dec 2024 (3 comments)
Jank is now running on LLVM IR - https://news.ycombinator.com/item?id=42276672 - Nov 2024 (16 comments)
Jank development update – Moving to LLVM IR - https://news.ycombinator.com/item?id=41845669 - Oct 2024 (49 comments)
The Jank Language: LLVM Hosted Clojure - https://news.ycombinator.com/item?id=32493217 - Aug 2022 (79 comments)
This is a genuine question, not meant to dismiss the project!
ETA:
Sorry, further down the article answered my question:
> “jank is also a good fit for any Clojure devs who want a lighter runtime without sacrificing JIT compilation, as they would if they used a Graal native image, or if they want easy access to native libraries for whatever reason.”
1. You get a native binary without sacrificing interactive programming. With a Graal native image, all of that goes away. With jank, you can still REPL in, change things, JIT compile code, etc.
2. You get Clojure -> JVM level of seamless interop from jank -> C++. I am pretty darn sure that this will be unprecedented, given the challenges of the native space (no standard reflection, differing ABIs, C++ templates, etc).
Aside from that, jank is making some huge quality of life improvements over Clojure JVM. I've shared some of this already, but I'll have a post coming out EOQ which will demonstrate the difference in compiler errors between the two. It's night and day.
I've been debating trying my hand at making a simple game engine...it would be great if I could use a Lisp to do it.
Last time I checked, they also allowed literal C code inline in Lisp programs. That's a bit more than the other Lisps which offer FFI.
It is built into TIC-80 by default, making it already useful for (small) games.
More detail would be appreciated, I'm not aware of any non-transpiling language that actually support full C++ RTTI/exceptions interop.
In particular, whether it is restricted in practice to types that are std::is_trivially_relocatable.
https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2024/p11...
As a clojure(script) developer of 10 years, I still try and avoid anything JVM related. Not that I have anything against the JVM — but I never did any Java programming, and to really learn about the JVM you have to learn quite a bit about Java… which is just very low on my priority list.
I have been dreaming about jank a little bit, so maybe this is a good place to ask the question, since I see the jank developer is reading the comments: would it be possible to write a module in jank that can be used in a Swift/iOS application (currently or in the future)? I assume so, but I’m not sure how accessible the outputs of jank are to other c libraries.
The reason I ask is I have an offline clojurescript front end. If I want native mobile apps, that means I either have to duplicate the logic, figure out some JavaScript bridge, use react native, or use dart — none of which seem ideal. Ideally I could just extract the critical business logic into a few modules and generate the header files for swift, import directly into cljs for web, and use as regular clojure for android.
Totally understandable if this isn’t a valid use case for jank, but it’s what’s captured my imagination :)
Jeaye, in contrast, seems to be taking a more deliberate approach in designing the language, not only taking the best ideas from Clojure but also trying to improve upon them.
Any good pointers to existing libraries to get a feel for the ecosystem?
What some call bloat, others see saved effort not reinventing wheels.
Or, I find a lot of script runtimes in games have a strict boundary separating them from the native code (often for good reasons), but I wonder if we could integrate more tightly given static analysis has come a long way.
Anything less than seamless rawdogging would be neutered C++.
My main objection to C++ is that in 2025 its not a language I would want to learn or use, nothing about its inherit qualities
I'm starting with C++ interop but will aim to provide interop with other native langs going forward, thanks to LLVM.
Once you used lexical scoping, you're not going back to dynamic scope either, the same goes for the ability to reason about "where an object currently is" lexicographically.
I think Carp (https://github.com/carp-lang/Carp) is a much more interesting contender for a native clojure, simply because it has lexical ownership.
I used clojure professionally for a number of years and I can't remember ever really using it.
I.e. if you're not tracking ownership in new-lang, you're probably doing it very wrong.
Lexical scope allows you to reason about scope on a syntactical level.
Take this code:
let x = 5;
fn foo():
return x \* 2
fn bar():
let x = 10
return foo()
fn baz():
let x = "hello world!"
return foo()
bar()
With lexical scope, you can look at `foo` and you know immediately what it returns, the behaviour of the function is completely local to it and is a syntactical property of the program.With lexical scope you just don't know what's returned, for all you know x might not even be a number, but could be any type that just happens to be brought into scope.
Ownership is similar, in that you make the existence of a value a local property. If a 'thing' is inside a variable then you can move it somewhere else, but it also means that it is no longer in that variable. And that tracking of where what is, is a syntactical property, just like with lexical scope.
Clojure has software transactional memory. With ownership it wouldn't need half of the machinery (only that for rollback):
let blocked_accounts = Set()
let account_bob = new ExclusiveBankAccount(200)
let account_alice = new ExclusiveBankAccount(600)
// ExclusiveBankAccount cannot be copied or cloned
fn transfer(source_acc, target_acc, ammount):
if source_acc.deduce(amount.copy()):
target_acc.deposit(amount.copy())
fn block(acc):
let acc_identifier = acc.identifier.copy() // identifiers can be copied and are not exclusive
blocked_accounts.put(acc)
return acc_identifier
fn unblock(acc):
return blocked_accounts.take(acc)
fn do_invalid_stuff():
let good_account = new ExclusiveBankAccount(200)
let bad_account = new ExclusiveBankAccount(200)
block(bad_account)
transfer(bad_account, good_account) // <- this will fail because bad_account was moved by the block function and no longer exists here, it's invalid syntactically
fn do_valid_stuff():
let good_account = new ExclusiveBankAccount(200)
let bad_account = new ExclusiveBankAccount(200)
let ident = block(bad_account)
let restored_account = unblock(ident)
transfer(restored_account, good_account, 100)
The above code makes sure that:- you can only transfer funds between two accounts in a thread safe way
- you can only transfer funds between accounts that are not blocked
Simply by not allowing something like this on a syntactical level you get a much cleaner understanding of what your code does. After a while you're wondering why we allowed anything else in the first place.
let x = thing();
let y = x; // the thing is moved from x to y here
print(x)
The fact that automatic memory management falls out of this is almost accidental, if you can track where a value is at any given time, you can also track the references to it, and the resources it uses. But that is not what truly makes it amazing, the fact that you can mentally think about digital objects as if they were physical objects is.It's interesting you mention STM. That's one of the reasons I only dabble in Rust instead of switching over to it completely.
In do_invalid_stuff(),
block(bad_account)
transfer(bad_account, good_account) // <- this will fail
This can only be determined syntactically if you and the compiler agree to stay on a single thread, right? I would expect STM to be the thing which safely bridges across threads - since it will realistically a customer will call transfer() but a bank manager will call block().No. This will fail because in block the ownership of bad_account is lost. You’d need block(&bad_account) or block(&mut bad_account) to pass a reference while letting the transfer succeed. The only way the transfer would succeed silently like that is if Account implemented Copy (the compiler would automatically inject block(bad_account.clone())). This has nothing to do with threading (which Rust would still have protection mechanisms for). It would work identically if the block and transfer were member methods (i.e. bad_account.block(); good_account.transfer_from(bad_account)) since even member methods can be declared to be requiring ownership.
Right, but all the promises of the type system have to lead to a first-class multi-threading experience at some point.
The safest concurrent program isn't a single-threaded program.
Parent mentioned them above, argued that that ownership supersedes them,
>>> Clojure has software transactional memory. With ownership it wouldn't need half of the machinery (only that for rollback)
but then only showed a single-threaded demo of ownership. I don't need STM either if I'm only on a single thread.
STM's a good model. You just write code as if it were single-threaded, slap an 'atomically' around the lines which shouldn't be logically divisible, and you're basically done.
But "aquiring" them just looks like having them all in local scope, so there is no explicit `lock`.
In the transactional case you mentioned there can still be multiple transactions having access to the same values, so you need the explicit transaction semantics. Ownership never has such cases, unless you use locks to manage ownership between threads, but it could also be done via channels, or CSP, or whatever.
Put simply: How would two thread get access to the same account at the same time? There would be a point in time where the object would have to be split to end up in both of them, which would be invalid.
This is why you see things like this in rust
let x = 5;
let thread = thread::spawn(move || { // <- important bit
println!("{}", x);
});
println!("{}", x); // <- this would fail
The thread uses a 'move closure', which takes ownership of all values from the outside scope that are used inside its body.Note that a value needs to be marked as `Send` so that it can pass across threads, but most types are.
That's what makes it so powerful. You know that any value you have in any variable, is only in that variable and nowhere else in the "world". Sure that value can be `Copy` and automatically copied, but that's still a new value. Your value can be `Sync` which means that a reference of it is `Send`, but you're not splitting up the value, you're creating and tracking a new reference value that is then send across the thread boundary.
(This is also why I walk past Erlang. Such emphasis on share-nothing prevents an account from being shared by a customer and a bank manager.)
So, what is the actual solution for a thread asking if any other thread has blocked(acct1), so that transfer() can proceed or be rolled back?
I’m not aware of any language that provides these kinds of guarantees statically or what that would even look like. You’re asking for a lot vs where the cutting state of language design is.
The way I solve situations like this is, by making sure that all accounts are moved to the same thread for a short period of time, which then handles the transaction. Which I find a lot easier to think about. It's essentially sharding by account and flexibly moving the shards into the same single writer context.
But if you want to do things the "old fashioned way" Rust provides plenty of classical synchronisation primitives like `Mutex` or `RwLock`.
You can also combine both approaches, e.g. have a single Mutex<AccountPool> that you can check out accounts from.
let x = 5
do_something_with(x)
do_something_with(x) // <- boom compiler error
you simply cannot use the same variable twice, because you lost ownership of the value it stores on the first call.But that's a bit cumbersome, so people add an operator that allows you to derive a new value from a variable without taking the old one. Let's call that one `&`. It takes a variable that would normally only have been allowed to be used once (because using it removes the value from it), and returns a "thing" that borrows the contents of the variable out, and behaves a lot like the original thing, but not quite.
let x = 5
do_something_with(&x)
do_something_with(&x)
And because those derived things can be tracked and only used once just like the original object we can make sure that no two things can derive such borrowed things at the same time in a way that is mutating the original thing. Let's call the operator that only allows one mutable borrow to exist at a time `&mut`.The ability to track that only one of these can exist at any given moment for any value is based not on some special properties of "borrows" or "references", but because of the ability to uniquely track ANY value.
Carp is a very neat project, though it's not a Clojure. It's just Clojure-like. jank is Clojure.
My point was that ownership has become a language feature that I expect, like lexical scope has become a language feature that most people expect.
It complements persistent immutable data-structures extremely well, especially in terms of performance via automatic transient-ication, and if you're operating in the native space I can't imagine that performance isn't on top of the list.
So in that sense, great language, please consider evolving beyond Clojure (archaic?) ownership model, because I'm pretty tired of Rusts bloated syntax and lovecraftian macro system.
Doesn't the focus on immutable data make most of the ownership stuff redundant? My understanding was that ownership lets you be sure that you're the only mutable owner of a resource, or otherwise that you share the resource immutably. If data is (almost) always immutable, which seems to be a founding principle of Clojure, then what does ownership/borrowing buy you?
You are right in that immutability or more precisely the persistence that copy-on-write brings somewhat solves many of the issues that ownership also solves.
Persistence allows you to not care as much about who owns what, because whenever something is modified it is simply copied (with structural sharing), so you know that whatever you have is local anyways, not because you are the only one who has exclusive access to the thing, but because you make a copy of it at the point where answering that question would matter.
But you miss out on two things from that:
- Performance: Those copies are cheap but not free. Borrowing complements _persistent_ immutability in that regard extremely well, because it allows you to check if you're currently the only one using something and modify it mutably in that case. Clojure has a concept called transients, where you create an intermediately mutable version of an immutable thing to perform batch updates, but you still have to manage that yourself, with ownership it happens magically behind the scenes.
- Exclusive types: The other comment I wrote contains an example of this. I have an `ExclusiveID` type in my Database that is similar to a `row_id`. I can recover a lot of the capabilities of transactions and software transactional memory from these, simply because a user can rely on `ExclusiveID`s existing only at a single point at a time in their code. And `ExclusiveID` itself is a completely immutable thing, but the "contract" on how it can be used is what makes it powerful.
Immutability on a domain logic level also does not help you when it comes to dealing with inherently immutable things like devices and networks, whereas ownership does.
So I would argue that those two concepts are complementing each other extremely well, both implementation wise, but also in terms of a the mental model of how your code works.
Being immutable isn't some "best-practice" like eating your veggies, where the more you eat, the healthier you are, and occasional sugar is OK.
I either can reuse the result of getFoo() or I can't. If getFoo() is 99% immutable - as the caller, it's my job to consider taking locks and/or making defensive copies.
If I want to publish library code that calls getFoo(), now my callers will need to consider taking locks and/or making defensive copies.
Your question was about Rust ownership, and I answered about Haskell immutability - but they're both the same in that they're pretty strict compiler rules, and you have to stray into unsafe territory to violate them.
Java interop breaks things of course, much like unsafe Rust breaks guarantees of that language.
getFoo() = getFoo()Immutability guarantees if I call getFoo(x) I can trust that nothing happens to x (assuming x is proper Clojure data and not some unsafe other thing like a Java Object). And getFoo can likewise return anything without there being any risk that anyone else modifies that thing. It does not matter who owns x or who owns the return value from getFoo, since no one is able to modify those things anyway.
* Database request was a bad example as, as far as I know, there is nothing built-in for supporting that, so that would already be in unsafe Java territory. But there are non-pure functions built into the Clojure standard library that getFoo could call to return different results on the same input, like slurp or deref, and probably many other ones (I have not used Clojure for a long time).