I built a garbage collector for a language that doesn’t need one

Implicit arguments! I'm more and more convinced that this form of controlled dynamic scoping dynamic scoping should be embraced by more languages.

> create a CPU architecture where pointers are actually (object, offset) tuples

Segments might yet see a renaissance (and CHERI might be indeed be a form of it).

Though arenas are useful even outside of "hacks".

In the end, a pointer (or a reference, even) is "just" a usize index into the heap, which can be thought of as a heterogenous collection.

I definitely think a language that has room for abstractions that aren't zero cost, while still having a borrow-checker makes sense. Using rust to make that language also makes sense. I think implementing a vm or interpreter entirely in safe rust would effectively force borrow checking on the resultant language wouldn't it?

Ruby seems like a good place to start the experiment. I'd prefer it end up with something more like julia or python just for my own use cases.

If OCaml/StandardML/Haskell/etc had anything close to the ergonomics and convenience of cargo for testing, setting up projects, adding deps, deploying them to any OS/platform, I would probably write about 30-40% less Rust.

I'm not saying we should add GC to Rust - but I think there is space for a Rust+GC language (or just improved ecosystem and tooling for other functional langs)

But that doesn't mean Rust with GC wouldn't be a nice language! I don't know of any language that really fits that description.

In fact, I'll pick just four things I like about Rust: a good type system, a supportive, productive community with a focus on doing real stuff, enough mindshare and buzz not to be irrelevant, and being not totally insane.

The first criterion rules out all untyped languages, Go, and the blue collar languages like Java and C#. The second rules out all existing functional languages. The third rules out whatever niche language you're about to suggest. The fourth rules out C++.

There really isn't a mainstream managed language which has managed to thread both the design and community needles the way Rust has.

Rust as a midpoint between Haskell and C, so really C with a nicer type system, combined with momentum/meme status is more important today than it's memory safety story. Memory safety just isn't the buzzword that it was even in 2019 let alone 2015.

> And you can probably bang out one yourself if you want.

I've written my own dataflow analysis code, aimed at lifetime checking, too, but that's not really the point is it?

Garbage collectors usually also make assumptions in the other direction (a non-conservative approximation) when it comes to finalizable resources (e.g., file descriptors). These uses do not involve the reference that is visible to the collector. This gives us Reference.reachabilityFence(), and again mistakes are blamed on the programmer.

No, s/he's not. The original claim was:

> It is in general undecidable whether an object is UNREACHABLE. [Emphasis added]

Being unreachable is not the same thing as being "no longer used by the code". The latter is indeed undecidable, but the former is not.

It doesn’t help you write a garbage collector to think of it that way. It doesn’t provide insight into the problem. The problems are elsewhere. Focus on reachability, dynamic understanding of the heap, performance, etc.

If new Object() truly doesn't have any effects visible other than the valid reference, then it's creation can be moved to after the if statement.

But I can list a few thoughts. The actual memory model is (seemingly) a flat address space, in which processes allocate segments, and everything else arises from there. There's some remapping behind the scenes, some segments are shared, some not, but this is beside the point.

And a pattern I see often is arena allocation, where you allocate a large segment and then "virtually" allocate smaller segments inside, down to individual scalar variables. This can happen several times with a lot of memory, but tends to be shallow.

For this and many other reasons, I see mutable memory is best structured as a tree, a shallow tree, almost relational, but still permitting nesting, where every item has one owner, one caller, and anything else is routed through that tree. Of course once this semantic model is established, you can reduce and optimize many calls into more direct calls. But you get clear intercept points when you need to decorate, block, remap and so on behaviors in the system. No "action from distance" surprises.

Everything should be pass by copy. For larger structures, we need copy-on-write optimization. Which means underneath the mutable tree there is a Directed Acyclic Graph of immutable segments supporting copy-on-write.

This is also not new, it's how forking works on Unix for example. Every modern OS utilizes copy-on-write on the file system, in memory, it's also utilized, if we think about it, in network caching systems. But to the process it looks like normal flat address space, where everyone owns their own copy of the content.

For collecting unused segments in COW you'd naively need refcounting. There are other approaches that borrow from GC algorithms, in a much simpler and more performant way, because, well, there are no cycles in a DAG. The best part is that once you decide to use copy semantics, you have a plethora of algorithms at your disposal to try and switch between adaptively (including... just copying the thing if it's small enough, which turns out is the fastest approach), while maintaining the same exact semantics for the programmer who doesn't have to care how it works under the hood.

Another benefit of copy semantics is that things are copied between processes, or especially between machines in a network, and now your language locally has the same semantics as when it talks to the network, which removes barriers and unifies interfaces. "Write once, run at any scale" if you will.

Anyway, this is not comprehensive. And just my opinion. But the problem with graphs is that it is the data structure with least discipline, and most complications. And in most cases those complications do not pay off. Not only they require complex GC, but they result in heavily entangled code, tons of shared mutable state, that gave the whole OOP space a bad name (that it doesn't deserve). This was never a problem with objects, because objects were never supposed to share mutable state. It's a problem of graphs and handles, a system that pretends everyone "has" a mutable piece of state... which it doesn't have, as ownership must be exclusive to be reliable.

Discipline of dependency management (packages etc.) often removes cycles and results in a tree or a DAG. This is not coincidental. And there's no reason we can't replicate this at the lower levels to individual objects.

Of course, we still need graphs. But not everywhere, and not all the time.

You can also easily reproduce a graph when you need it, by using simpler data with computation on top which builds the graph "on the fly", without it having to be your baseline memory model. You can for example easily describe a graph in two SQL tables: edges and nodes. But SQL state itself is not just any graph, and for a very good reason.

You probably know that the first databases were hierarchical and graph. SQL won not because it gave more freedom, but because it removed freedom in a highly specific way, which enabled larger more reliable systems, contrary to naive expectations. Too much freedom is freedom to shoot yourself in the foot, whether you know better or not.

Today graph databases have enjoyed a renaissance, but despite they have solid uses cases, they'll never be the baseline type of database. They won't replace relations. People have learned their lesson in databases. But in programming languages, they haven't. It's one of those things where we can't translate knowledge from one area to another, because the words we use for the same effects are different and therefore we can't make the connection.

I wrote more here: https://news.ycombinator.com/item?id=37122934

That being said I have observed that using Python as a web/backend language seems to be more common in North America than in Europe, but I have no evidence to back that up.

Why? Excitement doesn't always follow correctness or even "goodness" (whatever that means in context), even if we'd like it to. Fashion and marketing matter more when it comes to excitement.

I worked very little with D, and a lot more with Rust. I cannot really speak to the qualities of D because, by and large, I dealt with a very few instances of it, and only at the interface level (mostly writing in another language). But, in terms of fame and popularity -- Rust developers and backers put a ton of effort into promoting the language. There were times when every other week there would be an article in some popular tech. media source from all kinds of "big names" about how Rust is superior to something (usually C++). There were massive efforts made to create a self-sustained community of Rust-evangelists who'd go and spread the word to the heathens...

I haven't been around to witness the birth of D and don't remember if it had any kind of adoption campaign, but from what I can tell, even if it had a campaign, it didn't succeed.

Now, here's another example from the same category of languages: Ada (esp. with Spark). I'm not very knowledgeable about it, but from the little I know, there are a lot of good reasons to prefer Spark over Rust. At least, on engineering merits, they should be roughly equivalent. But, in terms of popularity, Rust beats Spark by a lot.