Thoughts on Rust vs. OCaml

---

When you allocate nodes in an arena, instead of getting back something like a Box<T>, or Rc<T>, you get a plain Rust reference--&'a mut T or &'a T.

Obviously, the first benefit is that these work well with pattern matching, field projection, and other useful language features, They are also "zero cost" in a way that Box, Rc, and Arc are not--everything in any particular arena gets freed at the same time, so there's no need for precise ownership tracking. In fact, &T in particular is Copy, aka freely duplicable, so you can pretty much treat these &T references exactly like garbage collected pointers in a functional language!

Arenas have some other benefits, too. They have great locality of reference, are tightly packed (almost as tightly packed as a vector), and in many cases let you allocate slices in addition to single references (so even if you think you need a Vec of nodes, you can still often stick with pure arenas; and slices are, again, Copy, meaning they work well with pattern matching and other parts of the language).

In the context of writing a compiler, arenas have another extremely useful property: they make it easy to efficiently create recursive data structures (particularly immutable ones, but even ones with cycles if you are willing to use interior mutability like `Cell<T>`). These recursive structures can use ordinary Rust references in the exact same way you would managed pointers in a GC'd language. The only restrictions are that whatever nodes you point to should be allocated in the same arena (or older), and a more complex type-related rule for safety that in practice doesn't hurt much.

The exact details are a little complicated, and it involves some subtle areas of the compiler, so feel free to skip this part (the tl;dr is that these rules usually apply). Essentially Rust can reason as follows:

* For safety, all references to values within a TypedArena must be accessible for at most as long as the TypedArena itself.

* Before the arena is dropped, its nodes are always accessible, meaning there are no restrictions on how they can be used.

So the problem case only occurs when not only do the node types stored in the arena include references to other nodes within the arena (more conservatively, with the same lifetime as the arena), they also have destructors that actually follow that field. It turns out that this almost never occurs in practice, so in theory virtually all Rust types should be usable with cycles; but the language can't reason directly about field usage in that way at the moment. Instead, Rust provides three mechanisms to make all this work out:

* Many types (all Rust primitives, many zero-sized types, and other "plain old data") simply lack drop glue altogether. Notably, regular Rust references (&mut T and &T) fall into this category, but the "smart" constructors that are annoying to use with pattern matching (Box<T>, Arc<T>, Rc<T>) do not. This is by far the best kind of type to use with arenas because it maximizes their strengths--as there is no per-element drop code to run, Rust can do no per-element processing whatsoever on destruction, allowing effective use of arenas to equal or even surpass the speed of the young generation of a generational GC in the right circumstances.

There are some arenas that even go further, exploiting the fact that with no drop glue you don't need to know what types are allocated to define an arena that lets you allocate heterogeneous types into the same structure--the only restriction is that the types must be `Copy` (a proxy for not having a destructor, in this case, as any type which has drop glue is prevented by Rust from also implementing `Copy`).

* Many other types in Rust have drop glue, but their own destructors are automatically generated by Rust. The Rust autogenerateD destructor code is very straightforward and just recursively calls drop on all the subfields / variant data, so as long as all of these are known not to use any fields with the lifetime of the arena, we can assume the parent type doesn't, either.

* Finally, there are the cases of Rust types that do have a destructor, do have a field with the lifetime of the arena, but don't try to follow that field within the destructor. While in reality this probably encompasses virtually all remaining types, Rust can't currently deal with lifetimes that are in scope, but not alive (though this is a useful concept in some other scenarios too). And in practice, the most likely scenario for encountering this involves using a type like Vec<T>, Box<T>, Rc<T>, Arc<T>, etc. inside an arena, where T satisfies one of the other two rules, but the outer container doesn't (since it has a destructor).

As Rust can't verify these scenarios itself, instead it provides an unsafe escape hatch, #[may_dangle], which can be applied to a lifetime that isn't dereferenced in the destructor, or to a generic type parameter that is only used in order to run its destructor manually somehow. Using this attribute in a `Drop` implementation makes the implementation require `unsafe impl` and can only be done on nightly for new types, but it's implemented for all the types I just mentioned, and more besides. This means that Rust immediately knows that Box<T> is safe to use in such a cyclic arena as long as T is, and same with Arc<T>, Rc<T>, Vec<T>, etc.

So ultimately what this amounts to is, that if you put recursive types in an arena, nine times out of ten it will "just work" and basically feel like you've turned Rust's ownership rules completely off, as long as the arena can manage to stay in scope.

Obviously, there's no free lunch, and you can see that this approach gets a lot less attractive if you're talking about storing things with unpredictable or highly irregular lifetimes (in an object lifetime sense), or if you want to package things up as owned data, or be able to allocate them in a function that doesn't accept a context (that library ecosystem is indeed great as you mentioned, but if you start using weird enough patterns it stops working so well). But my experience, having done this sort of thing for a long time now, is that the tradeoffs for arenas are usually worth it for at least some parts of a compiler, both for aesthetic and performance reasons. Their relative lack of popularity is, I think, more a product of poor marketing than anything else (as you can see, the arena story does fit into the borrows, ownership, and lifetime paradigm, but not in a straightforward way).

But the latter is pretty subjective; for instance, if you know it's safe to drop something when the program ends, but it might be safe to drop much earlier, in some contexts it still might be worth dropping at the end rather than dealing with the significant overhead of something like Rc or Arc. The reason I recommended arenas is that I find in compilers, often you have cases where you just need to define a scope at some point in your program way higher than when you create the value, and that's what they excel at--letting you initialize stuff with whatever parent scope you want, not arbitrarily restricting you to the function boundary.

    struct Point {
        x: i32,
        y: i32,
    }

    fn add(p1: &Point, p2: &Point) -> Point {
        Point {
            x: p1.x + p2.x,
            y: p1.y + p2.y,
        }
    }

I'm not quite sure what you mean by "the lifetime of a function"; lifetimes are associated with references, but not with functions themselves. In particular, it can be totally valid to return a reference to something that needs to last beyond the current function call; lifetimes are used to enforce that you'll get a compiler error if the memory your reference points to won't survive long enough. For example, it's totally valid to have a function return a reference to a field on a struct as long as the struct was also passed in by reference:

    fn x_val(p: &Point) -> &i32 {
        &p.x
    }

    fn main() {
        let p = Point { x: 1, y: -1 };
        let x: &i32 = x_val(&p);
        std::mem::drop(p);

        // Without this line, this will compile, since the compiler can tell that `x` isn't 
        // used after `p` is dropped. However, since `x` is used here afterwards, the 
        // compiler will generate an error stating that `p` does not live long enough.
        println!("{}", x);
    }

Also: https://www.purescript.org/

"Web development" is a grueling land war that requires massive investment in foundational libraries and fairly sophisticated tooling just to do a modern "hello world" app. Any language/platform that isn't explicitly dedicated to "web development" will always be lagging many generations behind those that do (i.e. rails and waves hands javascript).

I agree with the sentiment that Rust and OCaml aren't really peers. More generally, the urge to transition to Rust in general application development is as puzzling to me as it is to your parent here: it has unique benefits for systems programming, but that absolutely carries costs for a ton of non-systems contexts where you could just as easily use OCaml, or (modern) Java, or ...

They didn't have aggressive marketing or zealous community.

They didn't have massive influx from ruby rails or javascript webshit crowds.

Can you tell us a bit more about those cases? I've been writing front-end stuff using BuckleScript (and the OCaml syntax, which I for once really like!) for 6 months, and I haven't (yet?) stumbled upon those kind of pain points, I'm definitely curious!