ClojureRS – Clojure interpreter implemented in Rust
github.com
github.com
From what I understand the vanilla JVM implementation isn't built around a small core. So it's not like porting a Scheme. All the data structures and what-not are written with Java (not a criticism, it's great for performance and interop)
But wouldn't it make more sense to first rewrite Clojure itself in some small/minimal scheme-y subset of Clojure - and then port that subset to Rust/Dart whatever? (With some performance penalty)
I'm not super on top or Clojure's implementation, and people are prolly not doing this for a good reason - so I'm likely off base here somewhere
Actually indeed I think this is a possible route I went or have a note to try somewhere. The idea being if we can make a very lean Clojure core, and then the std library built upon that instead, it should be easier to port to other systems and share code. Gets tricky of course once you deal with code that relies at all on any sort of interop somewhere, but I should probably not get too much into it as my heads not been in that space for a while haha. The idea of a unified, universal Clojure of some sort (whether possible and practical or not) is an interesting problem in all this
Not that any of these are the individually the wrong choice at all. It does suggest a lot of duplicated work, but I also get the sense that each's goals are different enough they maybe don't have as much overlap as it's easy to assume.
The reason Clojure Classic is hosted on the JVM is because Rich wanted access to the Java ecosystem (and probably because the JVM is an awesome piece of tech). If ClojureRS follows the same mentality, the point here is someone wants to program in Clojure but have access to the Rust ecosystem.
From this perspective it is clear why they'd avoid taking a small-subset approach with a possible performance hit. This isn't a tour of force showing how great Clojure is & how clever someone is that they can implement it on Rust. The idea is to get work done, using Clojure, which may need access to Rust's libraries. It doesn't need to be elegant, that isn't especially important.
How about Scheme?: https://github.com/takeoutweight/clojure-scheme
Note that this project is unmaintained and the last updates are ~10 years old.
So bootstrapping is now easier.
That said, you likely would want native data-structure instead of interpreted code that implements them.
What I still continue to find interesting is that Idris2 compiles to Chez Scheme, and is somehow faster than Idris1 which compiled to C. The Chez Scheme implementation very much natively handles all important data structures.
But the design (granted I'm judging from the docs and API - not digging through the source) seems like things weren't designed for multiple hosts in mind.
A lot of the standard library functions are already written in Clojure itself, and don't use interop, but a lot of them, as well as types, and core data-structures are implemented in Java, and core functions that uses them tend to use interop. So that stuff needs to be ported as well.
And that's about it.
Now, you might think it would be great if all the types, data structures and therefore all core standard functions could also all be implemented in Clojure on top of the special forms. But I actually think it's specifically because that's not the case that Clojure gets ported to so many hosts.
The special thing is that this design makes Clojure hosted, which is very different from simply being implemented over multiple hosts.
For example, Clojure always uses the host's native types. So Clojure types are the same as the host it is running on. That's why then interop becomes a first class thing, and why Clojure always feels very symbiotic and compatible with the host.
And because it uses the host's native types, it also can't implement it's core data-structures and some of its core standard functions without using interop. Which is why those need to be re-implemented to be host specific normally.
I think there's a few extra things you could lift to be in pure Clojure, and I think Clojurescript has lifted more of them to be so, but still the types are same.as the host.
If this wasn't part of the design, the appeal of porting it would also be lost. It would just be like a bunch of exactly same Clojire language just with various quality of implement and performance characteristics, and people would probably just pick the ine that has the best performance and most stable.
But instead, Clojure becomes an alternate language for a multitude of hosts. Suddenly it means you can use Flutter as easily as with Dart, but have Clojure as your language to do so. It means you can use NodeJS as easy as with JavaScript, but using Clojure as your language, etc.
It's also why it's a bit wrong to say Clojure gets ported to other hosts, in reality it doesn't, what happens is someone creates a new dialect of Clojure that is well adapted to a new host. And that's also why it's so useful.
Correct me if I'm wrong, but you don't typically interact with the host directly when using core data structures. If you were to rewrite those for instance in terms of lists (like you'd probably do in a Scheme) I don't think that that change would be visible to the user from pure Clojure code. And similarly for the standard library. Then host to host you could re-implement them with whatever data structures you want.
This would in principle make adding and then maintaining a new host significantly easier
As for the types, are you speaking about how numbers are Longs/Doubles, and arrays are Java Arrays? And that matches what Java uses - so things work smoother (except when you get a BigInt..)？ I get what you say about changing that host to host improves the coupling - but you loose the ability to share libraries which is a shame - hence why you say `dialect`. I guess if you write a Clojurefoo for a new host that normally uses Ints and Floats then it's a design choice if you want it to match the Host or to match Java. If Java is close enough I can see it making sense to create a compatibility layer and have users deal with whatever friction ensues..
From my own limited experience, host interop is super important and a huge part of why I used Clojure. Calling out to Java libraries saves me zillions of hours of coding. But the actual interop code is also not a huge fraction of the code I write. If it were a bit more complex on a different host b/c I needed to fiddle with some typeas I don't think this would be a deal breaker. On the other hand, not having access to all of Clojure's libraries would be a disaster
I wish I knew this stuff a bit more thoroughly to discuss it more concretely :)
You could probably, but persistent thread-safe data-structures are a bit non-trivial to implement. And I'm not even sure you can build them atop a simple list, and even if you could, definitely they would neither be performant nor memory efficient.
I think it would be a cool project to have a kind of Clojure implemented over a more minimal core, like an implementation of the data-structures over a simple array or something, but it also wouldn't really be practical in usage I believe.
> This would in principle make adding and then maintaining a new host significantly easier
It would, but I wonder if it would just encourage toy implementations. In some ways, now it forces you to re-implement from the get go all the pieces that matter for performance and efficiency.
> As for the types, are you speaking about how numbers are Longs/Doubles, and arrays are Java Arrays? And that matches what Java uses - so things work smoother
Yup, numbers, strings, regexes, threads, namespaces, they're all host specific. Namespaces are particularly important, you want a Clojure namespace to be a module of the host.
Having the same types makes interior a lot smoother, and also it makes implementing Clojure to a new host a lot easier as well, because building custom primitive types over an existing runtime is non-trivial.
> but you loose the ability to share libraries which is a shame
That's why you've got reader conditionals, so you can still technically implement portable libs, though it does force you to re-implement the parts that differ as the library author.
But this is a tradeoff, either reusing the host libraries is trivial and easy, or reusing Clojure's would be. I'd say 90% of the appeal of Clojure on a new host is leveraging the new host's libraries, so I think Clojure makes the right trade off here.
And the IO pieces you'd always have to reimplement as well. And because of Clojure generally using interop for that, the person who ports Clojure also has less to re-implement, since users can just use interop.
> On the other hand, not having access to all of Clojure's libraries would be a disaster
How many of those libs are interop free though? Because otherwise you'd always lose access to them anyways.
Also, Babashka will probably always support more features than ClojureRS ever could, particularly the interop with the various Java classes/functions, as that'd be very hard to achieve in ClojureRS.
[1] https://github.com/hiepph/dotfiles/blob/master/scripts/_pock...
I’ve mentioned a few days ago [1] about the lack of a clean and consistent lisp (in the development sense, not in the core-language-is-tiny sense) with a CL-like REPL-driven development workflow: maybe Clojure with conditions and restarts can be part of the picture?
(In Common Lisp, restarts and conditions are a major part of the REPL as it allows the REPL to have a mechanism to catch error conditions and allows the user to resume execution appropriately, after the user debugs and fixes errors.)
There seems to be some efforts of adding user-land conditions, see for example https://nextjournal.com/pangloss/a-tutorial-on-conditions-an...
One could imagine another user-land REPL that also makes use of that, to give a similar REPL for Clojure as for Common Lisp.
Regarding picking it back up, would you have any interest in being funded by the community to work on the project? I know myself and others are very interested in seeing Clojure in Rust with interop, and would be interested in funding the effort if it's something you're interested in.
Maybe you could leave some contact details on your about page here on HN (no one except HN mods/admins can see your email) and we could talk privately if you prefer that.
It does allow errors to be caught by the Repl and choose how to handle them.
Honestly though, I haven't found the interactive restarts at the REPL useful. It's way faster to just re-evaluate the code again.
Maybe in CL they are more useful because the language is full of mutation? I can't say, I've only ever done Clojure and Emacs Lisp, but in Clojure I tried that library, and found picking options at the REPL to handle the condition was more laborious than just fixing whatever or adding a catch and pressing Ctrl+Enter again.
1: To create a version of Clojure that gets to exist independent of a particular platform
2: To explore a Clojure that, being more independent, has
Proper tail call elimination
Conditions / Restarts
Any other feature it might not normally be able to indulge
This is exciting stuff!
Then I did a very brief search for clojure libraries, things like parsing html. Most of the github projects were not seeing much activity (like last commit in 2020).
I've found bash-as-glue of unix programs like ImageMagick, curl, jq and pup very practical [2]. I was just looking for a possible alternative to be sure I'm not missing out. Doesn't seem like it.
Anyone wants to hop in and sell me on his own approach?
Where the shell work dips directly into areas clj is strong- pure data processing- I have recently been thinking that maybe bb could make it easy to create a full clj repl environment, so I wouldn't have to suffer through jq. But that wouldn't be doing the work in a shell with bb. It would be doing the work in a repl. FWIW.
One other point- lack of activity on clojure libs is not the red flag it is in other languages. There's much, much less of the churn and grind that there is everywhere else in the ecosystem. When there is something novel that doesn't have a native clj implementation, it very likely will in Java, and pulling that into clj- even straight from java source on github- is easy.
HTH. Cheers.
It's hard to beat this workflow. Like, what possible ""java"" library is gonna beat the ubiquitous unix tools?
For a commercial project I have been dipping into webassembly, and figuring out how to run wasm inside a JVM. The setup ergonomics are terrible, because it's native code land.
But it works, and performance is spectacular.
With all the build tooling targeting wasm, maybe there is a crazy future where native engines become pervasively available under wasm, and therefore inside a JVM....
This is not far from where Python has been- for 20 years. Its mechanical sympathy with native libraries is a big part of its current supremacy. But Py is a poor shell and repl, and most cli tools don't ship as engine libraries.
So nothing yet competes with bash on the file system chaining together executables.
Should there be widespread wasm, though, in THAT world, a Clojure repl with a nice menu of engines, caches, stores, all just functions and data- those ergonomics might get some people to switch...
(fantasy goggles off now)
Except that once you try EDN, you wouldn't want to deal with JSON anymore. EDN is a lot nicer - it's compact, doesn't contain "syntactical garbage", and easier to read and modify. To deal with API requests I use github.com/borkdude/jet, another useful tool from the author of babashka.
So to be clear, you can easily use ImageMagick, curl, jq, pup, etc. See: https://clojure.github.io/clojure/clojure.java.shell-api.htm... and https://github.com/babashka/process
Other benefits are that you only need to learn Clojure and suddenly you can use it for everything, backend apps, frontend apps, scripts, etc. You don't need to learn bash, js, java, go, make, etc.
> Then I did a very brief search for clojure libraries, things like parsing html. Most of the github projects were not seeing much activity (like last commit in 2020)
This surprises everyone, but those libraries still work, have no bugs, are missing no features, and can be used without issues in production.
Clojure is one of the most stable language, so things never break and almost never need updating.
People have a kind of Stockholm syndrome I think coming from other languages that if something didn't need a bug fix in a year it must be abandoned and broken.
And the reason you often don't need to update those libraries to keep up with the environment, like OS versions, is because they all leverage existing runtimes under the hood like JVM and that's the one that updates. So they're all secure and kept up to date, working with new OS and new architecture for free as JVM updates. This applies to Babashka as well, because it is implemented using a JVM.
If anyone feel interested, please give a try.
One thing to notice is Calcit is using indentation-based syntax while supporting macros. It's strange, but it works really well.
Edit: On to your year in review posts now.
https://jiyinyiyong.medium.com/year-review-of-2019-on-coding...
https://jiyinyiyong.medium.com/wip-2020-working-projects-38d...
Seeing Cirru being used is a good sign. Showing a tiny graph of usage over time can give confidence to new users.
I have several more articles/videos posted in Chinese, not ready to push all that forward yet... there are too many repos under Cirru and Calcit, many of them are not well maintained I'm afraid. :D
Would love to see this implemented as a macro too, like macro_lisp!