Weak pointers in Go: why they matter now | Hacker News Reader

Weak pointers in Go: why they matter now

victoriametrics.com

78 pointsvalyala33 comments

Is that WeakReference in Java?

Yes.

Java and most(*) other GC'd languages yes.

(*) though not all e.g. I think lua doesn't have straight up weakrefs, though tables can be made weak in key or value.

The article doesn’t actually explain how they work… How does the indirect pointer get set to nil? Does it use runtime.SetFinalizer?

https://cs.opensource.google/go/go/+/refs/tags/go1.23.4:src/...

https://cs.opensource.google/go/go/+/master:src/runtime/mhea...

Looks like most of it is implemented directly in the runtime.

How to shoot yourself in the foot

In the end of the article the author addresses some of the pitfalls of using weak pointers. You'll only shoot yourself in the foot if you don't fundamentally understand weak pointers and why you're using them in the first place.

"It's only a problem when programmers make mistakes". Gee, I wonder where I've heard it before

I lost track of the number of WeakReference bugs I've had to fix in a legacy Java codebase. Not looking forward to doing the same in Go.

As someone who has never used weak references in either language, what are the common pitfalls in practical usage?

The most common is if you create a reference to a field of a struct that is itself weak. The GC will gc the struct and your var can go from non-nil to nil under the hood despite having zero concurrent code (that you wrote).

The good news is the vast majority of code using Weak semantics is gratuitous - "clever" code that is actually broken premature optimization, and the fix is to just Don't Do That.

AFAIK the biggest issue is the nondeterminism ("inconsistency"), especially in languages which have a weaker type system and make upgrading easy (which is the case of both Go and Java)

- you need to remember to check the result for nil/null, but the type system won't tell you

- if you upgrade twice in a row, you can have both fail, both succeed, or the first one succeed and the second one fail

It can also be tricky to use them as building blocks for weak collections, if you want to avoid unbounded growth.

Why? You just have to understand how they work, and it’s not like they’re invisible.

You could make the same argument about writing in C/C++ and managing memory. You just have to understand how it works, right?

It’s still memory-safe, so not that big of a gun.

Unlike the language, which can segfault on race conditions with e.g. maps..

Don't forget panics on sending to closed channels.

/facepalm

panics are memory safe, unguarded concurrency can actually punch through memory safety entirely.

Oh I agree, thanks for clarifying it. I am just saying that it's a WTF for me.

I appreciate Golang's directness but they _really_ could have done a few things much better.

Go is considered a memory safe language.

Go without concurrency is fully memory safe. However, once you start using goroutines there can be memory corruption and segfaults: https://github.com/golang/go/issues/37484. Note that these are not recoverable panics (such as "writing to a close channel"), but causes completely arbitrary behavior:

    runtime: pointer 0xc00379ac60 to unused region of span span.base()=0xc001794000 span.limit=0xc001795e00 span.state=1
    fatal error: found bad pointer in Go heap (incorrect use of unsafe or cgo?)

    runtime: nelems=8 nalloc=5 previous allocCount=4 nfreed=65535
    fatal error: sweep increased allocation count

> but causes completely arbitrary behavior

I think we have different definitions of "arbitrary"

The moment you go off the memory safe route, you have absolutely no way of ever going back - the best case scenario is that the OS catches your wrong-doing and you get a segfault. Everything else might silently corrupt the runtime's state or business data. As they say, here be dragons.

Please elaborate! What's your definition? I was struggling a bit to phrase it precisely. My main point was to distinguish between "well-defined panic" (which is just like any other panic) and "silent memory corruption which keeps the program running".

To me it seems rather arbitrary that appending to a slice concurrently in two different goroutines causes the garbage collector's sweep-phase to internally think another allocation has occurred.

Some things are plain unimplementable in a nonleaky way without weak pointers (or even something stronger like ephemerons[1,2] or weak maps or the like), so at some point you have to face the music. Event / observer / dependency-graph / reactivity systems are one cluster of those.

[1] https://wingolog.org/archives/2022/10/31/ephemerons-and-fina...

[2] http://lists.squeakfoundation.org/pipermail/squeak-dev/2022-...

How do they intervene in the reactivity?

Genuinely curious as I have implemented one such system in Go. Maybe I could improve in some stuff.

I did use some kind of weakreference substitute (using an integer map) but that was because I was lacking a feature (some kind of dependent type).

I guess that wouldn’t be necessary if your dependency graph is static, but if dependencies can themselves change during change propagation, you do need weak references.

Imagine input A affects node B affects output C. Then you probably want B to be collected if C becomes garbage. So the strong references need to go C→B→A. But the way a simple push strategy would actually proceed if A changes would be that A triggers a reevaluation of B, which then triggers a reevaluation of C. But A could be either immortal for semantic reasons (e.g. “the left mouse button was clicked” can’t really become garbage) or perhaps kept alive by some entirely unrelated dependent D. So A needs to be able to refer to B if it’s still alive but must not hold B alive, and similarly for the edge between B and C.

The way to deal with this is to have store dependents in a weak set. (I don’t actually see how to implement one with plain weak references without finalizers.) Alternatively, just give up and say the dependency graph needs to be specified up front and only dies as a whole.

My own dependency-graph/reactivity thing is not ready to see the light of day, so if you want some references I guess I can direct you to the author of the Sodium FRP library (not to be confused with the libsodium crypto library) and an FRP book mentioning[1] he implemented a cycle collector as a workaround for the lack of weak references in JavaScript (at the time).

[1] https://web.archive.org/web/20240507181526/http://sodium.nz/...

Oh ok I see why now. I eschewed this by using getters (the reference part of weakref is being used when one doesn't want to use explicit getters).

Unlike reactish frameworks, I do not track dependencies but my reactive model is different and more fine-grained/lower-level. It's not needed.

Great in-depth article, unveiling a lower level of Golang previously unknown to me.

Also have to watch out for nil pointer dereferencing:

https://wakatime.com/blog/48-go-desperately-needs-nil-safe-t...

People imagining that “adding the missing features” is what Go needs should be paddled.

Well, good news, it's not a new feature at all:

> Interestingly, it used to be more of an internal tool, but recently there’s been a push to make it public

I think your anti-intellectual bubble should be safe.

I’ve left Go behind for some years now. I wish them all the best, but I do not approve.

Reply on news.ycombinator.com