...to give myself the opportunity to work on whatever I found interesting, without regard to outcome, commercial viability or the opinions of others. One might say these are prerequisites for working on Lisps or functional languages.
Having used Clojure professionally for a couple years now, I feel so blessed. It is really well cut out for a whole host of problems that are common but extremely labour-intensive in conventional languages.
Maybe one of these days I'll get time for my own sabbatical, and solve the two major tasks I see with Clojure: taking the performance from good to excellent; and hosting it in a robust systems language suitable for implementing well-trodden functions.
There have been a number of attempts to build Clojure on top of Go (I worked on one), and another on top of Rust. Both lack the dynamic runtime ergonomics that Clojure leverages. Clojure is also seeing significant performance benefits, both runtime and startup, from the Graalvm, but that also will have limitations.
I am wondering about D, which should be sufficiently low level from a systems perspective but may also have sufficient dynamic capabilities.
Do you have a link? The only one I'm aware of is https://github.com/candid82/joker.
Believe it or not, this is relevant to an ongoing sabbatical project of my own. I'm grossly unfamiliar with Java, so it's often hard to read through the Clojure implementation :/
The Java implementation implementation is hard to read even for Java folks, it is more of how a C++ programmer would write Java, both in terms of syntax, and semantics.
Many of the issues Clojure has with the JVM are shared by regular Java apps, so I think there is motivation to improve many of these areas over time, even if it takes a bit.
You may find this interesting: https://github.com/spy16/sabre
It's still pretty green, but it might float your boat.
AFAIK Rust ditched the optional GC long ago, but I think a lot of the infrastructure that has enabled async/await, in the way of hooking in a “runtime” through attributes and macros, could help with building a GC into a rust host. I think this could be a special strength if you combined the futures infrastructure with the allocation logic, maybe with a local spill/scratch pseudo-heap for allocations that don't escape.
Think ztellman's aleph and manifold interfaces, but based on tokio, with some of the allocation magic (or deallocation magic, depending on how you slice it) of BEAM.
I think I've been a bit let down by some “Clojure-inspired” languages which do not have HAMT or CHAMP datastructures, or do not have good implementations of them.
The JVM's profusion of high-quality garbage collectors is another thing that is hard to compete with; though I think there are massive opportunities with a collector similar to Shenandoah 2.0, if the allocator can be aware of the properties of Clojure.
Furthermore, a purpose-built runtime would be an amazing way to explore transparent non-volatile memory. I think there is staggering value in transparent access to datastructures that are significantly larger than memory, especially if that access is inside a future/task run by a NVM-aware executor, and can wait safely. Think MapDB, but persistent, immutable, indistinguishable from the standard Clojure datastructures, and integrated with the garbage collector.
Of course, as I've not written anything significant in Rust, I will shut up about it beyond this, because there's nothing more obnoxious than people extolling the virtues of something they've never accomplished anything significant with.
IMO one of the areas of abstraction leak in Clojure is around error handling. Most of the time the options- out of band and out of code path, like Exceptions, and in band, like null- don't sit well with me. I usually want in-band, explicit, with the capacity for some sugar so if necessary the error cases can be organized in one place. Comma-ok does this for me without any magic, and I would like to see it as a lowercase-p protocol integrated into things like the threading macros (there has been some work in this area...)
But high level, comma-ok is just data, not a type- another area of abstraction leak.
I hear the interest in arriving at a more efficient GC but don't have much current background in that area. I did a lot of GC work at the configuration level with the JVM, and read a bunch of the source a decade or more ago, but recognize that's not my area. I similarly spent a bunch of time in the linux kernel world 15 years ago, and still check in on the mailing list from time to time, but that's a different life.
Agree with non-volatile mem opportunity, though note that was the intent with virtual memory architectures and then CPUs went and ruined it with the cache hierarchy. From a performance perspective if your code is hot, cache-aware will noticeably outperform non-cache aware. Figuring out ways to make this automatically efficient at runtime in the context of also supporting GC would be a fascinating PhD project.
Appreciate the engagement, cheers.
Maybe take a look at Joker [0] to start. A Clojure interpreter written in Go.
I've never been more intellectually fulfilled, nor happier.
If only languages like Clojure, F#, OCaml, Kotlin, and many more enjoyed more popularity and support.