Hacker News Reader

Comment by iopq | Hacker News Reader

Finally some new ideas for AOT compiled languages that don't devolve to "what if we just have memory bugs some of the time?"

View on HN

Generational references leak if a counter reaches int_max and involve an increment on alloc and on free. Seems pretty close to reference counting to me.

https://verdagon.dev/blog/generational-references

Statically eliminating memory operations does seem to be a win though.

> Generational references leak if a counter reaches int_max

With 64-bit counters, that's never going to happen. Alloc/free costs more than a nanosecond, and there are lot more than one element that you will be allocating, but even if you somehow managed to reallocate the same object a billion times a second, it would take over 500 years to run out of indexes.

What about if you reach 100 billion times a second (maybe with 128 cores) and have a long running system? Maybe in this case 128-bit or 96-bit counters are better...

If your program is allocating so much that you can exhaust a 64bit counter, you have a seriously bad program plus a serious memory leak. Exhausting the counter would be the least of your worries.

Practically speaking you could never allocate memory that fast, a memory allocation is going to be well over 1000ns on average.

Then there's the little matter of address space. Pointers on x64 are limited to 47 bits, meaning that if even you had a magical memory allocator with no book-keeping overhead, and all your allocations were 1 byte, you'd run out of pointers first. The actual virtual memory space is limited further on many operating systems, but you're still always going to be well short of 64bits.

Memory is meant to be reused, "64 bit counter will be enough for everybody" is not how systems programming works.

Except when it is. You'd be surprised what systems programming looks like.

Reference counts are not re-used when memory is re-used. And again, even if for some reason you had a global 64bit counter that you incremented on every allocation and never decremented, and you could somehow handle a billion allocations per second, you'd have 585 years before that counter overflowed back to 0. No computer or program can run for that long.

> "64 bit counter will be enough for everybody" is not how systems programming works.

Except when it is: https://threatpost.com/another-linux-kernel-bug-surfaces-all..., fix at https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

VMAs sound expensive. Of course a 64 bit counter is going to work for moderately expensive things.

If you have a multithreaded app doing a lot of communication, that's going to be a lot of cheap allocations happening very fast.

Reducing GC and allocation overhead results in more allocations being done, and pushback against ever-expanding allocation behavior is more of a challenge. Instead of ten other things being a higher priority than judicious data architecture, it's dozens or more.

There is a separate counter for every memory object. One counter is only ever going to be touched by a single core at a time.

And even if there was a single counter, multithreading cannot make incrementing a counter faster. Two cores cannot write to the same cache line at the same time. Instead, cache lines need to bounce across cores when you write to them, and this takes such a long time that it turns the time it takes to roll over from centuries to millennia.

You can't allocate and deallocate the same address (incrementing that address's generation by 1 each time) 100 billion times per second.

Is it even physically possible to allocate memory that fast?

My understanding is that this has some benefits over reference counting, though it it similar. Part of the issues with reference counting is that they are shared, meaning that it needs to be atomically incremented and decremented whenever you make a new reference (for instance in C++ if you return a shared_ptr or similar). Generational references though instead track something as a part of the value you pass around, meaning that it has better locality and doesn't suffer from contention.

Copying references is assumed to be more frequent than allocating and freeing, so this is a win.

Typically that slot is disabled on overflow, so that no more objects can be created at that slot / memory location, which avoids the handle collision. The slot could be recycled at specific points in the code when it is certain that no more handles for this slot are out in the wild (not sure if Vale does that)

(Or possibly the whole region could be discarded once it was running full, and the physical memory recycled at a new virtial address. There's plenty of virtual address space to burn through when not limited to 32 bits)

Reference counting involves an increment every time a reference is shared. That's a lot more overhead than just on alloc and free - in particular, it involves going to main memory (or using part of the cache) a lot more in order to change the ref counts. Whereas on alloc and free, those frames have to fetched anyway (at least in my understanding of how memory allocators work)

Ref counts have cycles, which cause leaks (in this universe).

But I digress. Overhead like increments only matters on hot paths, which are very few. The Python + C stack for ML is a manifestation of this truth.

Having ergonomics of a “regular language” (affects all code) and the ability to optimize for performance (hot paths only) and stay in the same language is what I’m excited about.

Uh? While generational reference are memory safe your program still crash when there is a memory issue.. It's much better than going on in a corrupted state but still it's a crash.

Better than giving away all your secrets, a crash is great in comparison

I have a new favorite HN comment :)

Reply on news.ycombinator.com