Rust for Clojurists | Hacker News Reader

Rust for Clojurists

gist.github.com

151 pointslsh39 comments

Good article. He missed out the main pro though (and literally the only reason I ever use any Lisp): hot swapping. The ability to change code on the fly as the program is running. For this to actually work you need 2 things:

1. No explicit types: Everything in Clojure is just a list or hash map, so I can add extra members to anything at runtime no problem. The amount of static checks Rust does at compile time makes this non viable.

2. Fully immutable data structures: If my changes cause exceptions, then the program can just rewind and throw away the new state it was building, let me fix the error I made, and then continue on as if I never committed the broken code. In Rust an exception would mean I've potentially mutated state in a bunch of places, and now can't get back to the previous working state at all.

In which scenarios do you use hot swapping?

Just about every scenario? Any algorithm I am working on, any function, any config variable I want to tweak in real time, any GUI I am developing, and so on.

So for interactive work? Versus updating long running server jobs?

I'm used to using the REPL in F#, but you can't redefine data types and have old code pick up on it - only new code. Such an approach is theoretically possible in Rust. Debuggers can do this for C++ to some extent.

Clojure has a very REPL-driven workflow, where you connect your editor to a REPL (usually emacs+cider) and get instant feedback about anything, reload code on the fly, and inspect your program's state. This is a fundamental concept of pretty much any LISP, and is lacking in rust.

You don't need dynamic types to allow hot reloading, as long as a function has the same absolute path and type signature it can be done. I'm pretty sure miri[0] plus some compiler hooks would allow this, although it would need to reach nearly speed-parity with debug code if not release code. Hot reloading and/or repl-driven development is my number one programming desire right now, and Rust is my most-used language, so this would be life-changing for me.

Also, Scheme and CL allow free reign on data mutation and RDD/HS work fantastically in them

[0]: https://github.com/solson/miri

I didn't say it wasn't possible, but I've found that hot-swapping in languages that use mutable data structures, and have heavy type systems, makes for a poor experience. If I can constantly cripple the runtime with hot-swapping, or I can't change any type definitions, then there is little point to it.

Typed/racket supports hot swapping to a similar degree as clojure

Following on this, it should be possible to use something like CLASP [1] to do hot loading on C++ code objects.

https://github.com/drmeister/clasp

The BEAM and particularly Elixir satisfy all of the above., 0 is supported in the VM by releases, though admittedly probably not to your satisfaction. 1 is true in Elixir. Structs are just maps with a __struct__ key under the hood, though you can't arbitrarily add things here. I think this is true with defstruct in clojure, as well. 2 is true in Elixir, except via message passing between lightweight processes, which then return NEW state (of any shape) representing their current state at the end when you invoke their functions. And we have hygienic macros =)

Even C can do hot swapping using shared libraries, and it has none of those features.

I've seen a guy change some code for shaders, go into another tab and that tab already had the new changes after the compile. If you can do this in C++, then surely you can do stuff like this for Rust.

You can certainly have a hash map of trait objects in Rust (it just points to the vtable and the data). This means anything that has an impl for the Trait can be stored in the hash map.

Rust also doesn't have exceptions, using algebraic data types to hold on to the errors and return them up the stack. Because it's on the stack, it's not some global state that's mutated, but we just went back to the calling function and we can attempt to try again.

+1 for this gem right here. Well done!

"Many people try to compare Rust to Go, but this is flawed. Go is an ancient board game that emphasizes strategy. Rust is more appropriately compared to Chess, a board game focused on low-level tactics. Clojure, with its high-level purview, is a better analogy to the enduring game of stones."

Yeah, the wry humor throughout is great. Another good bit: "While the Rust project doesn't start off with any equivalent to Clojure's namespace declaration, once you move beyond a single source file you'll need to use it. This isn't C or C++, where you just include files like a caveman. Rust separates code into modules, and each source file is automatically given one based on the file name."

> Rust chose to call its [package] format "crates". This reflects the language's industrial roots and the humble, blue collar town of its sponsor: Mountain View, California.

Nice :)

The terminology there was actually inspired by the fact that Grayson Hoare would walk to work along the docks in Vancouver, passing the shipping crates as he went.

s/Grayson/Graydon. I blame autocorrect. :P

Some funny gems:

> You will remain a replaceable silhouette with no discernible identity.

> This isn't C or C++, where you just include files like a caveman.

>The upside is, you will curse it at compile-time instead of at runtime. The downside is, "exploratory programming" means exploring how to convince the compiler to let you try an idea.

After that it gets a bit more serious, but still a very good Saturday afternoon read.

> You're in an industry reaping disproportionate benefit from loose money policies, leading to a trend-chasing culture of overpaid nerds making web apps. You feel guilty about this, but there is nothing you can do about it because you have no other talents that a rational person would pay you for.

/r/2meirl4meirl

But in all seriousness, this was a great article and an exceptionally fun read!

Nice. My biggest problem with Rust is the poor performance and lack of complete & sane libraries, especially for HTTP.

I was running into serious issues with the following code:

https://gist.github.com/l1x/5678c0fdfc2c1a6034b8bcc3800de93c

Its performance is saturated around 60K req/s on a 32 core box with 10G networking while wrk was more than happy to do 3M req/s. I was trying to dig further into why it is slow and it seems that hyper is the culprit. However, I was not able to come by this issue and just used wrk2 for testing. Let me know if anybody has insight into how to go beyond this perfroamance or what are the issues with my code.

Hello! It's been a bit since I dug into hyper, but that's surprisingly low. Just to check, did you compile this with optimizations enabled, like `cargo build --release`?

Yes I did compile like that, even had optimization on.

[package]

name = "hammer"

version = "0.1.0"

authors = ["me"]

[dependencies]

hyper = "0.10"

[profile.release]

opt-level = 3

debug = false

rpath = false

lto = false

debug-assertions = false

codegen-units = 1

panic = 'unwind'

I'm not sure about performance vs wrk (don't kwow wrk), but just to give an idea: I noticed you're not using Tokio Hyper. Try Hyper's master branch, which fully embraces Tokio's async.

EDIT: From wrk's README:

> wrk is a modern HTTP benchmarking tool capable of generating significant load when run on a single multi-core CPU. It combines a multithreaded design with scalable event notification systems such as epoll and kqueue.

Your example is multithreaded but IIRC release Hyper isn't using event notification async, which Tokio Hyper does. It might help.

Thanks, I will look into this.

- Spawning a new thread for every request isn't good, you can reuse threads this causes a non-trivial Kernel load

- The standard channel library isn't bad but it's a far cry from great. You end up allocating every SEND. Which has its trade offs.

- You are doing a print in every thread which requires acquiring a global lock, and at least 2 system calls, and a global allocation.

- You aren't keeping your client/connection alive. So every new thread has to do a new 3 way handshake every request. 5 way if your is HTTPS.

Thank you, it would be useful to show how to this properly.

Answers:

- we are not spawning a thread for each request

The following means we are creating N threads and execute a loop in every one of them, unless I misunderstand Rust entirely.

  for id in 0..NTHREADS {
    let thread_tx = tx.clone();
    thread::spawn(move || { } ....

- I am not sure what coould I do about the standard channel library

- Doing a print at the very end once, the impact on performance is negligible

- keep alive used to be broken with hyper and I am not only looking for keepalive but pipelinening as well. Let me know if you are aware of a HTTP client in Rust that can do both

I could be doing something wrong, but I tried your code using hyper master here and got similar results: https://gist.github.com/anonymous/d5fa7a62ae1d3927878a976228...

Again I'm talking out of my ass here (very new to Rust's async) but AFAICT running core.run(work).unwrap() inside the loop is the same as just running it synchronously (unwrap is waiting for work to end, i.e. there's only a single concurrent connection ever per thread.)

It'd probably be better to continously run a core per thread and .spawn many tasks for it so that there are several concurrent connections per thread. Or maybe just .run a future which is the .join of multiple .get futures?

They are called __clojurians__.

While I understand this kind of guide (showing how things are done in one language/ecosystem vs another), is there such a thing as a Clojurist that would have benefited from it?

I'm not asking whether there are Clojurists. Of course there are.

I'm asking whether there are people that primarily and only know Clojure, and thus would benefit from a translation guide between Clojure and Rust, instead of say between C or Java or Python or Ruby etc and Rust.

I'd think that anybody who uses Clojure was already familiar and/or proficient in some other language before, much more major than Clojure.

Yes. Me. I've been programming for about ten years, but mostly jumped from language to language until I found clojure. Clojure is the only language I really "know".

I'm better at Python than Clojure, but I appreciated this article anyway. Clojure and Python take different approaches, so a comparison of Python and Rust and one of Python and Clojure highlight different parts of Rust.

I think it's satire?

No, it's written in a funny way, but it's a legit guide.

Not even a clojurist, but this is the best Rust intro I've seen so far.

Reply on news.ycombinator.com