How Go detects struct copies with sync.noCopy | Hacker News Reader

How Go detects struct copies with sync.noCopy

func25.dev

70 pointsfunc2576 comments

> noCopy is a special marker for types that must not be copied after their first use.

if it looks like a hack, walks like a hack, and quacks like a hack...

It's not really a hack. It's a hint to a static analyser, that's all.

Most languages would encode such behavior in a trait or a protocol instead of a zero-length struct field. It's type information and ought to be encoded in the type. From that perspective, I think it is a hack.

This feels like a style complaint and not one about substance. An inaccessible (because it's named _) zero-length struct field is just another kind of metadata. It also doesn't require you to pollute the method set, which would be a bigger issue.

The real hack to me is that anything which simply has Lock() and Unlock() methods is considered uncopyable.

Coming from Rust anyway (can't say I'm familiar with Go), nothing about having it as a trait would pollute the method set, traits don't have to have methods (e.g. pin, unpin, send, sync, etc.). Yes a zero length field can be metadata and this is a style complaint but a struct's fields are traditionally its data and the type is its metadata, I feel like it's harmful to mix these two as I at least wouldn't generally look at fields to find metadata for the struct.

Interfaces in Go (the closest equivalent to traits in Rust) don't have to have methods either, but they are structurally typed. This is like "static duck typing" if you will. So an interface with no methods is implemented by every type in the language (indeed, this became such a useful pattern that the name "any" was reserved for it in Go 1.18).

Marker interfaces can exist in Go, but here they are another kind of hack. They must define at least one method (which doesn't have to be public), though that method is never actually meant to be called.

I'm only peripherally familiar with Go. IIRC, Rust defaults to move-only structs with the option to make them copyable with the Copy trait. Swift defaults to copyable structs (like Go), but has a ~Copyable generic type constraint to make them move-only.

Rust has similar zero-sized-types, probably the most 'magical' being PhantomData.

> The real hack to me is that anything which simply has Lock() and Unlock() methods is considered uncopyable.

It's a shame they made them generic, taking no arguments and returning nothing; if they'd gone for `Lock(sync.LockType)` or something like that, it would be more difficult to accidentally trigger the `noCopy` flag.

Go's entire schtick is being simple, eschewing language complexity in favour of letting the programmer handle it themself. Like C, but with pointer safety and garbage collection.

We learned from C++ and Rust that languages can be so smart that people can't effectively use them. Go is the opposite. It's so dumb that anyone can use it, but it doesn't have some native language features you might want.

How is this simple? It's basically a hint to `go vet` that uses a special interface pattern. I'm baffled by what some people call "simple" lol

How much additional complexity do users need to be aware of when writing new code because of the way this was implemented? Adding new features makes the language bigger, for something that Claude tells me is used 20 times in total in the entire Go codebase.

I don't know the answer to those things, it feels like it's sort of on the one saying "This is simple" to explain why.

Because it doesn't expand the language and force the user to learn new features, it instead puts the ugliness deep in the library where nobody needs to interact with it.

This explicitly leverages a hidden "feature" of the language, which is how Sync impacts copying (rather unintuively). It leverages this to create an interface based on that hidden feature to create a marker that is consumed by a specific tool (not the compiler).

This feels like at least two layers of special behaviors.

What's "this", and what does the user need to learn to make use of it?

noCopy. In order to understand it, they have to learn that this is enforced only by `go vet` and that the marker relies on everything described in the `2. How go vet and noCopy work` section of this article.

Of course, they don't have to learn anything other than "use `go vet`, it's what enforces this" to use it. But that's true of almost anything?

It's the difference between trying to learn the language and trying to learn the implementation details of a library. Is the kind of crazy hack that Rust std uses to prevent TOCTOU attacks with symlinks a language complexity issue or not?

I suppose you can add features to the language and then tell people to avoid learning them.

> Is the kind of crazy hack that Rust std uses to prevent TOCTOU attacks with symlinks a language complexity issue or not?

I have no clue what you're talking about and I doubt that it's relevant.

The claim was made that this is "simple". It doesn't feel simple to me. I'm asking about the criteria used and how this fits into that to justify the "simple" claim.

The implementation details of this construct are implementation details of the language.

The answer is that the implementation details are abstracted away by the library, the same way that the implementation details of Rust std::file access are abstracted away. That means that while the implementation may not be beautiful, the language doen't need to expand in order to accommodate 20 uses of the feature.

The language could be expanded to support this, but then every user either need to ignore a part of the language, or learn it. Oddball hacks in the internals of a library don't affect users of the library. This article is a deep dive into how the internals of a library happen to work on this iteration of Go.

Soooo like a PhantomData in Rust?

Nah... I like Go, but this is a hack. Wiktionary --not the ultimate authority, I know, but still-- defines to hack as: To make a quick code change to patch a computer program, often one that, while being effective, is inelegant or makes the program harder to maintain.

I could accept having an anonymous embedding as a kind of syntax directive. So many languages have had directives bolted on later, so that wouldn't be a deal breaker. But then it should be accessible in all code and (external) code should be able to implement behavior as well.

I thought about it again, but you can of course use your own types in this very way. You can "test" if a struct has a certain (zero-sized, embedded) member. It's just cumbersome, and needs to be implemented through-out your code, so it's only suitable for very small "flagging" operations. It's not comparable to annotations in other languages.

Every language can't be rust.

I'd want to blame Go's compatibility guarantee for this, but I can't because it wouldn't actually stop them from adding a proper solution for this..

Just like the magic comments, it's a sad thing to see appear in this language because it feels like magic incantations one has to know that bend over backwards to not actually extend the language to fit the use case.

This shipped with Go 1.7, almost ten years ago to the day.

I don't think this changes anything about the point I was making.

Go's full of hacks, and holds no shame over it. Zero-initialized everything, and proceeding to 'defer' instead of RAII, generic builtin types despite lack of generics (until recently), no builtin list type, slices having capacity...

This was Go's design philosophy until Rob Pike left - to do the simple thing simply and not try to be clever about it.

> no builtin list type

Wait, which thing do you mean by a "list type" ? A growable array type like Rust's Vec<T> or C++ std::vector<T> or the ArrayList type seen in several languages ?

Or do you mean a linked list type akin to C++ std::list or std::forward_list or Rust's std::collections::LinkedList ?

"List" is vague, which is appropriate if you're talking about very high level abstractions where it doesn't matter how it works and 5 gigabytes, 5 bits, 5 weeks or 5 seconds are all finite so who cares - but in the real world we usually do care.

Yeah, I really don't see much of a reason for Go to get a first-party linked list type. In most cases in non-list-oriented languages the performance and ergonomics are awful compared to a competent growable array type, the mechanical sympathy of "real" linked lists is generally terrible.

Plus they're extremely easy to build if you truly have a good use for one, particularly with generics.

It doesn't matter which one you mean, because Go has neither of them.

It’s not simple though. The language is simpler, sure. But you pay for language simplicity with program complexity. In go, you have to write and debug a lot more code.

I don’t mind spending a few extra weeks learning a more complex language if doing so saves me months of time down the track programming and debugging. That is an excellent investment.

I've not seen this in my experience tbh, the extra code that Go requires is ugly but not complex, the lack of ergonomics actively discourages "clever" solutions and, as a result of this, people tend to write the kind of straightforward code that doesn't end up needing lengthy programming or intense debugging.

At my workplace we've used many languages over the years (C#, Python, Go) and Go teams are the ones that by far do the least amount of yak shaving and have the most intelligible codebases.

Yep. But with go 1.22/1.23 and later this is changing. It's becoming a hell of a mess like many other big languages. I think it was two consecutive releases back in 2024 where they added for ... range and generics? That was when I gave up.

Sad that the one language that managed to occupy that nice spot in language design space for an extended period of time, isn't doing so anymore.

Of course, you can actively restrict yourself to standard go, but not needing to do that was the whole point.

The support for iterators is relatively new, though being limited to data structures that an iterator makes sense on, they don't exactly get around in the language and pollute everything everywhere.

The support for generics is years old and I don't believe anyone who claims it has ruined the language. I've barely encountered them in the wild and I've never encountered the thing people were really worried about in the wild where something has 4 generic parameters that are themselves complicated generic parameters of other things. If you're encountering that, it is either some one-off library I've never encountered, or it's because you or your team are writing it, to which the solution is, stop that.

I'm not even sure I've yet seen a "generic" in a library in Go that isn't simply straight up a generic data structure, the core use case for generics. I've written a couple of such things but they're all internal code.

It's hard and annoying to read though. Constantly beating around the bush, circling the point but not stating it, not unlike LLM prose.

100% also our experience. We have an internal CLI which has grown to almost half a million lines of Go, mostly contributed to by first-time gophers (and agents nowadays), and with relatively little work spent on making sure the core entities and interfaces encourage doing the right thing, the entire codebase is still surprisingly readable and free of unexpected behaviors.

I’ve always felt like the lack of many complex language features nudges teams to also keep their code simple.

There is always two aspects to a language works in practice: how it formally works, and how the community uses it.

Go is (was?) simple, and also (encouraged by the language and influence from its developers) the community mostly aims to keep the usage simple.

It’s the lack of parametric enums (sum types) which really gets me. I use sum types constantly. Nearly as often as I use structs.

For example, in typescript you can define a json value as something like:

    type JSON = null | bool | string | number | [JSON] | {[k:string]: JSON}

Go forces me to reach for interface {}, and use a nest of dynamic dispatch code. It’s horrible. Go code is harder to write, harder to read and it runs slower as a result.

The decision is baffling. Especially given go now has generics, which are waaay more complex than enums. And sum types in go could be used to fix the constant (result | error) boilerplate. And remove nullability. Sigh.

I'm coming to think there's a selection effect here. The people that want to write complex code feel unsupported by Go and avoid it.

Yep. I'd much rather 50 lines of rust than 100 lines of go. I find shorter programs to be generally easier to read and more likely to be correct.

This is good for Go ecosystem!

So… what language do you use?

No (useful) language is simple. Claiming go is not simple without naming a language for comparison is a party foul.

Rust. Typescript. More complex languages let me write shorter, simpler programs.

I don't write haskell, but I understand the appeal. Haskell is this philosophy taken to its natural conclusion.

RAII has the advantage that you can't forget to do it, but defer has the advantage that you can handle failure in ways other than panicking. Of course, in many cases (e.g. closing a file), there's generally not much you can do anyway even if you want to handle the error directly, but at least it's possible.

Rust can handle that with RAII without panicking just fine. And not only Rust..

Sure, you can just ignore errors entirely, and this is what Rust actually does today, at least for std::fs::File. The only other option within RAII that I can think of is that you can mutate some external, longer-lived state.

The primary way to deal with error-on-clean-up in RAII languages is to not rely exclusively on RAII for it. Rust's File type, for example, has sync_data and sync_all methods (which, to be fair, only even need to be called for writable file handles). I don't think there's anything wrong with this approach, but it ends up being just as explicit and therefore forgettable as defer.

It should be noted that you can (at least in Rust) actually implement defer using RAII; see e.g. the scopeguard crate. Since RAII is block-scoped, this defer is also block-scoped (like Zig) rather than function-scoped (like Go).

Rust uses affine types, which means that the compiler guarantees that you clean resources (call the destructor) either zero, or one time. If you call it zero times, then the compiler inserts the call to the destructor for you, in which case there is no opportunity to handle errors in the cleanup, so the result is they are ignored (or you get some kind of panic)

A system that is based on linear types would have an advantage here. In a linear type system, the compiler guarantees that you always call the cleanup function (destructor) exactly once. With such a system, you can have the destructor return an error result, and since the call will always be explicitly written in the code (rather than generated automatically by the compiler), there will always be an explicit errorn handling code branch.

> (e.g. closing a file), there's generally not much you can do anyway

If closing a file fails then you treat it the same as how you would treat a write failure:

  int err = 1;
  FILE *f = fopen("whatever.txt", "w");
  if(f){
    if(5 == fwrite("Hello", 1, 5, f))
      err = 0;

    if(0 != fclose(f))
      err = 1;
  }
  return err;

Code which writes to files and doesn't check for errors on close is subtly incorrect, although my understanding is that kernel devs bend over backwards to make failure unlikely, probably because everybody does it incorrectly anyway.

Not that it matters, but fclose() doesn't happen in the kernel, so the kernel devs can't do anything about it. All libcs have essentially the same implementation:

- is fp NULL or already already closed? return error

- call fflush() and return error if it fails (fflush also happens in userland, it does a seek() then a write() of the userland buffer)

- call close() and return error if it fails

close() follows essentially the same process inside the kernel: check fd is valid, call flush() (this time truly to disk), close it.

More like Pike's design philosophy was "let's do some half-thought things and sabotage any current or future improvement proposal for decades to come, while gaslighting everyone that Go is good because Google and because people can't see difference between 'systems' and 'system' programming".

The whole Go design philosophy in one sentence, that is what one gets by refusing to adopt modern language practices.

If this was true, why did they add generics?

The generics that they eventually added, because of programming language market pressure, Kubernetes had an whole code generation gizmo to work around it, are a clunky implementation versus what could have been done if done from the get go.

The generics implementation is perfectly fine. The fact that you completely dodged my question is telling. But please, keep living in your alternate reality.

I did not dodge the question at all, you just didn't like the answer.

"Market pressure for language adoption".

Just like all the features they keep adding where they said that Go doesn't need them in first place, turns out those features exist in other programming languages for a reason.

This idea that they just caved and added them due to pressure is a false narrative invented by detractors like yourself. Here is a timeline if you're interested:

2009 Go released.

June 2010 https://github.com/golang/proposal/blob/master/design/15292/...

Jan 2011 https://github.com/golang/proposal/blob/master/design/15292-...

March 2011 https://github.com/golang/proposal/blob/master/design/15292/...

Oct 2013 https://github.com/golang/proposal/blob/master/design/15292/...

Dec 2013 https://github.com/golang/proposal/blob/master/design/15292/...

April 2016 https://github.com/golang/go/issues/15292

Aug 2018 https://github.com/golang/proposal/blob/master/design/go2dra...

June 2020 https://go.dev/blog/generics-next-step

Jan 2021 https://github.com/golang/go/issues/43651

Jan 2021 https://github.com/golang/proposal/blob/master/design/43651-...

Feb 2021 Proposal accepted.

March 2022 Ships in Go 1.18.

You could criticize Go for moving too slow. But I'm pretty happy with the way they landed. They definitely don't feel bolted-on.

Ah the usual barrage of links from Go folks, luckily we have also come prepared.

> In three years of Go surveys, lack of generics has always been listed as one of the top three problems to fix in the language.

https://blog.golang.org/why-generics

> In retrospect, we were biased too much by experience with C++ without concepts and Java generics. We would have been well-served to spend more time with CLU and C++ concepts earlier.

https://github.com/golang/proposal/blob/master/design/go2dra...

And naturally Rob Pike's pearl,

> think the transition to getting polymorphism into the language through generics is has still got some years to work through there's still things that don't quite work right

https://youtu.be/yE5Tpp2BSGw?t=2134

If they didn't want to add generics, why were they actively working on generics proposals and designs since the inception of the language? Was it a hedge against the potential of being pressured into adding them in the future?

Go survey results....

The survey started in 2016. But even if it didn't, that still wouldn't make a difference. You're clearly not a serious person, so I think I'm done responding.

Market pressure does wonders.

Seriousness is a hard topic among Gophers, what today does not matter, is apparently quite relevant tomorrow, and generics aren't the only example where this kind of pivots take place.

So whatever.

So, like https://doc.rust-lang.org/std/marker/index.html?

Those are integrated in the compiler and don't need a second tool to work.

Also, AFAIK, they only leverage standard language behaviour and are not a hardcoded special case.

Nearly everything in that module relies on at least some hardcoded compiler magic (what Rust calls "language items", with a `#[lang = "..."]` attribute on the definition). The only exception is `Send`—and even that is still an auto trait, which on stable Rust can only be defined by the standard library. (There are plans to also remove the lang-item status from `Unpin`.)

Fair point, I was only looking at the structs, not the traits.

I spoke with Aliaksandr Valialkin (author of noCopy) and he gave me his reasons:

- https://x.com/valyala/status/2088638160242683954

He also gave an answer of what he would change now: https://itnext.io/go-evolves-in-the-wrong-direction-7dfda8a1...

It seems he's not happy anymore with the new direction of Go because they're implementing things from other languages.

He's right. Countering the performance advantage of Rust and keeping its simplicity would be higher priority.

> I'd remove user-defined generics from Go, and all the overcomplicated shit related to them, including iterator functions.

Go advances one blub ragequit at a time.

This feels like it should ideally be something public in the structs package so anyone can leverage it, not just a specially blessed internal thing for the sync package.

https://github.com/golang/go/issues/70811 Go moves slowly

You can very easily define one yourself.

You can exploit the mechanism described in the article yourself, but it's already changed once in the past and is not part of any compatibility guarantee. As with structs.HostLayout, a blessed structs.NoCopy in the standard library could guarantee that it works forever. I think the bigger issue remains that it doesn't actually do anything in the language (but, then again, neither does structs.HostLayout--yet).

I agree, its a nice piece of semantics to be added to a struct

It's not a specially blessed type. As the article says, anything that implements sync.Locker acts like this.

Very good article. Interesting approach working in harmony with `go vet`.

Reply on news.ycombinator.com