Jank development update – Moving to LLVM IR | Hacker News Reader

Jank development update – Moving to LLVM IR

jank-lang.org

190 pointsrefset49 comments

Incredible work. A native Clojure would be a dream come true!

Wish jank the best of lucks. Hope I can contribute soon.

From their website, Jank is a dialect of clojure, and strongly compatible, I am not 100% sure what this means, but to me at least, it means it is not clojure and it is not native coljure

That being said, what would be the benefits of a native clojure, you have common-lips and guile if you want native

But again, Jank is not clojure .. just clojure-like, or so it seem

Clojure is language that was designed to be 'hosted' on many different underlying language runtimes, not just the JVM/Java. If Jank can implement clojure.core correctly (pass the tests!) and handle .cljc files then it counts as Clojure in the eyes of the community. See also ClojureScript, ClojureDart, ClojureCLR, SCI, Cherry, Electric, Rama etc.

** Clojure is _a_ language [...]

(apologies!)

refset is right, if this implements clojure.core it will be seen as a "real" clojure in the community, ala ClojureScript.

People use the Clojure that is the right fit for the environment they need to operate in, so arguments about authenticity don't tend to come up. Clojure has always been positioned as a pragmatic Lisp and the situation very much reflects that.

The OG Clojure is the one for the JVM and for some people this will always be the only "real" one in some sense, because JVM interop is a huge deal - it gets you access to tons of high-performance Java libraries. If you're writing server-side Clojure that's hard to beat.

But the other clojures have strengths in their niches. If you're writing for the browser, JVM interop doesn't buy you anything but JS interop does, hence ClojureScript. If you're doing a lot of shell scripting you'll probably want babashka for fast startup times. If you're doing mobile+desktop GUI development with Flutter you'll use ClojureDart (if you use Clojure at all).

Jank is aiming for C++ and I'll be curious if there is an interop story there. But again if you're looking to interop with an existing C++ code base it's sort of academic to ask whether Jank is "real" clojure or not.

> But again, Jank is not clojure .. just clojure-like, or so it seem

jank is Clojure. If it works on Clojure JVM and it works on ClojureScript, it should generally work on jank. Clojure doesn't have a language specification, but the community knows what a proper Clojure should feel like. Strong Clojure compatibility means anything outside of interop should just work. Naturally, interop on Clojure is very host-specific; Clojure JVM does one thing, ClojureScript does another, Clojure CLR does a third thing.

> That being said, what would be the benefits of a native clojure, you have common-lips and guile if you want native

At this point, we'd be discussing the benefits of Clojure over CL and Guile, which ends up being a different discussion. jank is for Clojure devs who want a native host and good native interop. It's also for native devs who want to introduce interactive programming and FP into their systems.

How complete is it?

I looked at github and came away with the impression it's very much still a work in progress, but you seem to imply it's closer to being complete?

Does nrepl work well? Can I just load a relatively arbitrary cljc file?

This discussion made me check it out as I'd like to use it in conjunction with some C++ files/libs, but looking at the project left me really quite unsure as to it's current state.

jank is still under heavy development. It is not released yet. A lot of features are working, or complete, but the language is at least several months out from being alpha released.

The majority of this is captured here: https://jank-lang.org/progress/

Thanks for the clarification, I'm looking forward to it's eventual release! =)

can you use jank in .cljc files and portable code regions just work? platform specific interop guarded of course?

Yes! jank has its own :jank reader conditional which you can use.

Maybe this has has already been covered, by I would not target LLVM IR in 2024.

I'd target MLIR (like Mojo does).

1. It's a much easier/better target to work with.

2. It's a strict super-set of LLVM IR.

3. Much better optimizations are possible that are specific to your language.

Separately, I'd love to have a Clojure-friendly interface to MLIR—whether via Jank or something else.

This is a great suggestion. I've been doing some research on Mojo, but didn't realize MLIR was the recommended approach going forward. I'll look more into this. Codegen is pretty straightforward, from jank's AST, just using LLVM's IRBuilder, so I'd expect migrating to a superset shouldn't be an issue.

Is LLVM IR a stable target these days? I once heard of a project that got bit pretty hard with LLVM IR interface changes in the (not all that recent) past.

Absolutely not. They do invasive changes all the time. At work we maintain an LLVM compatibility library that provides a stable interface across different LLVM versions. It has grown quite big over the years. The pure C interface is in comparison much more stable, but also quite limited.

Not just the API changes, the LLVM IR syntax can change drastically as well. E.g. not too long ago they switched from types pointer types to opaque pointers. `i32*` just becomes `ptr`. [1]

[1] https://llvm.org/docs/OpaquePointers.html

IR can be auto-upgraded between LLVM versions.

citation? i work on LLVM (like llvm/llvm-project) and i'm not aware of such a tool.

The textual IR is not backwards compatible, but the bitcode format has been best-effort auto-upgradeable for as long as I've been involved (2006). The policy is documented here: https://llvm.org/docs/DeveloperPolicy.html#ir-backwards-comp...

bitcode policy i'm aware of - it's the "IR" part that caught my attention.

a pretty evident case of not almost got caught

As long as you use the C++ API, the pace of change is reasonable.

If you try to keep by by generating textual IR, it can be a nightmare.

I do agree that the pace is reasonable. Both with opaque pointers and the new pass manager API there was a quite a long transition period where both were supported before the old system was deprecated/removed. But with every new non-patch release, some part of the public API is changed and we have to adapt.

> But with every new non-patch release, some part of the public API is changed and we have to adapt.

bumping only on major release is recipe for pain and misery. everyone project i'm familiar with bumps weekly against HEAD. much more manageable.

I think the bitcode parsers (binary and textual) are quite backwards compatible. Old bitcode IR will be upgraded transparently. So you might not have to keep up.

I love this writing style, very easy to read. Best of luck with the optimizations effort! looking forward to hearing more about this project.

Thanks for the feedback on the writing! Stay tuned. :)

I'm curious what advantages Jank has over GraalVM, since Graal also supports interop with llvm languages.

Jank should be able to produce significantly smaller binaries than Graal native images at the very least, but the potential for performance gains looks to be rather large, e.g. per "jank has been consistently beating Clojure in benchmarks" [0]

[0] https://jank-lang.org/blog/2023-07-08-object-model/

That makes sense that the potential for performance improvements is greater, I'm just a bit skeptical it can actually "consistently beat clojure" in anything except for micro-benchmarks. It's mentioned in that article that jank uses a pretty basic GC while the JVM/Graal contains multiple state-of-the-art GCs tailored to different use cases. It's been shown many times that the JVM or Graal's JIT compiler have better peak performance over a long duration than compiled programs. This simply comes down to the fact that there are a host of specialized optimization strategies only possible when compilation can happen during runtime.

> there are a host of specialized optimization strategies only possible when compilation can happen during runtime

That's a big part of why Jank is interesting to me, because it can properly embrace the idea of runtime JIT using LLVM. The JVM may still have an edge with profile-guided optimization for the foreseeable future, but perhaps Jank can help close that gap and support even more advanced possibilities for data-intensive workloads, e.g. along the lines of https://www.lingo-db.com/

With Graal native images, JIT compilation is not an option any longer. The runtime is static. With jank, you have a native binary, but you can JIT compile code with it and interop seamlessly with the native world. jank will also have an option for static runtimes, with no JIT compilation support, which can perform direct linking, better inlining, and whole program LTO. These builds will be smaller and lighter than native images.

Already, jank is faster than native images in the benchmarking I've done. The largest benchmark, so far, was a ray tracer.

Maybe there are legal benefits because it's not from Oracle?

IIUIR, Clojure uses the JVM garbage collector to clean up memory.

How does Jank do this whilst keeping the user code… still Clojure?

Jank uses Boehm GC (written in C, also used by the likes of Inkscape, Guile and Mono) - it is discussed a little in the previous blog posts, e.g. https://jank-lang.org/blog/2023-07-08-object-model/

Guile seems to prepare to switch to https://github.com/wingo/whippet - a GC C library that can wrap Boehm/BDW but also provides more (and less) advanced GC options. Should be a good option for Jank and similar projects as well - especially if they already use BDW.

Have you considered MPS? https://github.com/Ravenbrook/mps .

Yep! MPS was considered, but Boehm was significantly easier to get going. For jank's next GC, the plan is to use MMTK.

I'm just an enthusiastic bystander so I don't know the answer. Hopefully Jeaye can chime in.

This is a wonderful project, I'm very excited for it. I look forward to reading about the WebAssembly plan

Great that it is already making use of C++20 modules.

does this mean there's a path from clojure to wasm ?! and the wasm component model ?

Yes! Since jank is using LLVM, AOT compiled jank programs have a path to WASM. This isn't a primary goal for jank, but it's one we can definitely pursue once jank's Clojure parity and general dev UX has been established.

that would be amazing!

How useable is jank already? Is it production ready?

Not production ready. Not very usable right now. I need several more months to improve the tooling, error handling, and general compilation functionality. Stay tuned!

Awesome work Jeaye!

Nice to see a language take developer UX seriously and focus on good compilation times. I wish this was more common.

Fast compilation is undoubtedly more essential in the Lisp paradigm of REPL-driven, interactive programming with S-expressions than in other languages. A pre-requisite perhaps for the world to 'Stop Writing Dead Programs' [0]

[0] https://jackrusher.com/strange-loop-2022/

Fast compilation is needed in all languages.

Nothing is as annoying as waiting 5-15min for compilation of c++ code

Reply on news.ycombinator.com