Comment by Animats | Hacker News Reader

Comment by Animats | Hacker News Reader

Parent Full thread

Hm. That's an interesting approach. The constraint reference encapsulating a ref counter is useful.

The "destructors with parameters" are interesting, but it's not clear what prevents using the object (or trying to) after destroying it.

None of this seems to address locking. The trouble with locking in C-type languages is that the language has no idea what data the lock covers, and thus it's hard to do an analysis for race conditions.

"We made two tiny classes, Request and RequestHandle. Each had only a pointer to the other. When one was destroyed, it would reach into the other to null out the pointer, thus severing the connection."

I've wanted something like that for Rust. Rust has problems with back references in trees and lists. There's no safe way to do those. What you need for a back reference is something like what Vale has there. The back reference has to be a non-owning optional reference. The only allowed values which can be assigned to it are the owner and None. Changing the ownership implicitly sets it.

(This is one of the two big safety holes in Rust. The other is partially initializing an array. There's no way to say to the language that "0..N of this array are initalized". So "Vec" has to be unsafe. If both of those holes were plugged, there would be far fewer excuses for "unsafe")

Thanks!

There are three things preventing someone from using the object after destroying it:

   * If it was an owning reference, the compiler will ensure nobody else uses it.
   * The program will halt if there are any live constraint references to an object we destroy, preventing anyone using one of those.
   * If it was a weak reference, the language will (conceptually) null that out when we destroy the object.

Locking goes deep into Vale's concurrency story, but I can try to sum it up:

   * By default, an object can't be visible to two threads, every thread has its own isolated region (similar to Pony and Rust).
   * We can use a mutex to contain a region, and Vale's "region borrow checking" can make sure references don't escape the lock's duration.

I'm curious about that second safety hole Rust has. I don't think Vale has that same hole, as we initialize every array with a size and a lambda, and there's no opportunity to stop it halfway through (well, our form of "exceptions", involving pure functions or rollbacks, could stop it and keep everything safe), you can see an example in our little roguelike sample [1] on line 415.

[1]: roguelike.vale, https://github.com/ValeLang/Vale/blob/9541d414ce6b1c2745de19...

I'm curious about that second safety hole Rust has.

All variables are supposed to be initialized before use. But a Rust Vec, like a C++ std::vector<T>, has both an in-use "size" and a usually-larger pre-allocated "capacity". This allows adding elements without getting a new memory block.

Implementing Vec safely in Rust runs into the problem of initializing the capacity not yet used. Not initializing it is unsafe, so Vec has to have "unsafe" code.

I'd argue for this:

First, data types should be divided into "fully mapped" and "not fully mapped". A fully mapped type is one where all bit combinations are valid. Fully mapped types don't have to be initialized to be memory-safe. This includes integers, floating point on most systems, but not booleans. (C++ calls this "plain old data", or POD, but allows Booleans and enums. Whether Booleans with values other than 0 or 1, or out of range enums, should be considered "plain old data" is a good question for a language designer.)

If it's not fully mapped (worst case, it has a pointer or reference in it), what do you do about uninitialized data in the unused "capacity" area? If it's not initialized, you have a pointer with a junk value, as far as the language is concerned. If it has to be initialized, you have a problem if initialization requires an input value.

So I'd argue for the concept of a partially initialized array as a language feature. For simplicity, only being initialized from the beginning to a limit (0..N) should be supported. (Yes, there is such a thing as a giant sparsely initialized hash table where allocation info is stored outside the table. See Google's sparse hash table. That's very rare.) The idea is that you designate some field of the structure as the "size", and accessing beyond "size-1" is a subscript error. But you can initialize at "size", and this increases "size" by 1. You can decrease "size" if you own the whole object.

If you have both of those features in a Rust-like language, you should not need "unsafe". Except to call other languages, maybe.

I don't think you can have a "fully mapped type" the way you describe it. Uninitalized memory in today's compilers do not have an observable value.

https://www.ralfj.de/blog/2019/07/14/uninit.html

The compiler’s representation of an uninitialized lvalue might not be observable as the article describes, but an uninitialized heap allocation from the OS (i.e. malloc()) is observable by design.

Of course a language and its clever implementation can make reading unintialized values from malloc()‘d objects UB with all that implies, but that’s a spec choice.

So I believe you could define what’s returned by malloc() as safely castable to one or more “fully mapped types”, which would solve the problem for vectors at least of those types.

Yes, you need to be able to at least do that, for big I/O buffers and such. Those are usually raw arrays of bytes, so no problem there.

The problem I was discussing is with growable arrays, where the memory underneath is partially initialized.

That's interesting! I'm fairly certain Vale avoids the problem with its lambda approach, but I can see a partially initialized array being very useful for Rust. In fact, I think Cone [1] does exactly that; you can designate a certain field to be the size for another an array field.

I haven't looked at google's sparse hash table yet, I'll have to codesearch that =)

[1] https://cone.jondgoodwin.com/

In fact, I think Cone does exactly that;

Interesting. Cone has every known approach to memory management. GC, refcounts, arenas, pools, single ownership, thread local... Worth an experiment, but you probably don't want to go there for a widely used language. Think of the library compatibility problems.

For locking purposes, there's an argument for thread-local, shared, and transferable objects. Transferable objects are what you put in a queue between two threads. They're single-thread and don't need locking when in use, but can be handed off to another thread.

It's useful if transferable objects are transferable by copy. This is useful for machines where not all memory is shared. We're headed there; cache management gets harder and harder as the number of cores starts to approach 3 digits. So think ahead here.

This is really a very cool language! I wonder how nested regions works out in practice. That seems like an obvious way to deal with Rust’s limitations on aliasing (IIRC Cyclone used regions that way) but I wonder what the pitfalls are in actual use.

Thanks! We're really excited to see how they work out too, there's some exciting potential in there.

I can imagine having programs which are tree-like hierarchies of regions, but I'm starting to suspect we can have directed acyclic graphs, and perhaps even very small cycles in them...

Reply on news.ycombinator.com