Good union types in Go would probably need types without a zero value
utcc.utoronto.ca
utcc.utoronto.ca
Make the union type default instantiate as the first possible type of the union. You might think this is a pretty dumb design decision that can lead to programmer error, but it meshes perfectly with the other mistakes in the language.
The standard implementation is pretty simple: the different variants of a sum have a tag that tells the runtime which variant they are, and hence where to find pointers.
Go already has a "type switch" for interfaces, so it must already store tags with values in some cases, which is exactly what is needed to make this work.
So this is not the right solution I'm afraid.
Perhaps there is some confusion around terminology. What OP calls union types are known in type theory as sum types, and are actually different from what type theory calls union types. Type theory union types are like set unions.
Zero values don't make any sense anyway. As the original comment said "You might think this is a pretty dumb design decision that can lead to programmer error, but it meshes perfectly with the other mistakes in the language."
That won’t work in Go, because it would not be safe. You would need to coordinate the change with the concurrent garbage collector. It’s not designed to do this. Redesigning it to do this would be non-trivial.
Go already has a lesser-known safety problem similar to this involving interface types, but that safety problem is at least avoidable if you write code without data races in it. With the union safety problem, the data race is harder to avoid.
Previous discussion here: https://news.ycombinator.com/item?id=42305999
If you avoid the race, forcing sum types to only ever contain a single pointer, sure that avoids the race too.
It still doesn't get you a meaningful zero value, which the language design assumes to exist for every type.
Other commentators have pointed out that nil is a perfectly good default value for a union (if it's underlyingly an interface).
[1]: https://en.wikipedia.org/wiki/ML_(programming_language)
If it was truly the case, there wouldn't be any PL research anymore. :o)
In fairness to the Go developers I don't think they ever said they were against generics; they just put off adding them for ages because they didn't have a design they liked. I imagine the same could happen with type unions - in 20 years when everyone has stopped using Go they'll come up with a reasonable design. :-D
Why are we not using OCaml or standardML or even F# (which I tend to look favorably on) more widely then?
> Why are we not using OCaml or standardML or even F# (which I tend to look favorably on) more widely then?
In my opinion there are a number of factors. I have most experience with OCaml, so on that:
1. Terrible Windows support.
2. Poor documentation.
3. An obsession with linked lists and recursion, which are great from a theoretical point of view, but abysmal from a performance point of view (and also simplicity IMO).
4. Poor syntax. The dearth of brackets and semicolons makes it very hard to visually parse. Something as simple as mismatched brackets can be very frustrating to resolve. The insistence on (a -> b -> c -> d) style function types is unnecessarily confusing. Generally when there has been a choice between academic cleverness and accessibility they've always chosen cleverness.
5. Global type inference is pretty clearly a mistake at this point.
IMO part of the reason Rust is so successful is that it has taken a lot of the very good ideas from ML and basically fixed all of the above issues.
Standard accessible C style syntax, but expression based and with proper ML style types. Fantastic documentation and Windows support. No linked lists to be seen.
It's such a compelling design someone copied it for Go: https://borgo-lang.github.io/
Rust has its fair share of corners and weird syntax.
It also has missing features so to speak.
Go evolves at a more cautious pace, and is actually more successful. To the extent that rust also borrowed from Go.
The point is not even comparison but I'm still unconvinced about the whole, ignoring everything.
Minimalism, caution, and starting from first principles with a clean slate is obviously the right choice.
Yeah I guess because it is really good.
> Rust has its fair share of corners and weird syntax.
True, but IMO they're in smaller bits of the language, like trait bounds. Not something as basic as blocks, function calls or variable declaration.
> Go evolves at a more cautious pace, and is actually more successful. To the extent that rust also borrowed from Go.
Go is more successful because it is older. Give it a few years. I actually don't know of a single feature Rust has borrowed from Go. It is not even listed as an influence:
https://doc.rust-lang.org/reference/influences.html
> Minimalism, caution, and starting from first principles with a clean slate is obviously the right choice.
Throwing away learnings from other languages is pretty clearly a bad choice IMO. It would be like creating a new language where everything is nullable... oh.
What? There is plenty of interesting ongoing PL research (e.g. a "perfect" solution to combining subtyping with type inference was only found quite recently). But some languages are happy to ignore results from 50 years ago.
You don’t just drop a union type into a language with some syntax and semantics. There’s an impact on how the runtime works. Every language’s design is somewhat impacted by runtime considerations. For Go, the runtime consideration is the concurrent garbage collector, and the data race that you would have between the garbage collector and the union tag. You have to solve that problem in order to have unions in Go in a way that would make sense in Go.
Go unions would not look like unions in ML.
You could trivially implement unions in Go by having union values be pointers to structures, and it could be a different structure for each variant. This is basically just sugar around "any" and typecasting. The problem? You’re now allocating for every assignment.
In this case, if I understand your complaint correctly, what you are saying is there is a race condition between setting the tag of a variant / union / sum type and the GC reading that tag to interpret the memory layout. I agree this is an issue, but it's an issue with any memory write. This is why garbage collectors have a write barrier, and the Go GC is no different (e.g. [1]). So ultimately I don't see the problem as unsurmountable. Yes, it will require some thought and some runtime changes, but the foundation (i.e. type tags, the write barrier) is already in place.
[1]: https://ihagopian.com/posts/write-barriers-in-the-go-garbage...
edit: to be complete, design decisions, even 50 year old ones, are not absolutes, but trade-offs and subjective. Eg, is a GC more important than a highly-expressive type system? Depends who you are. But a simple language, even one with a minimal type system, can be highly productive.
That’s not a great example because there are plenty of languages with expressive type systems and GC. F# for example.
At this point there's probably more GL bindings for Rust out there than active developers.
If you haven't figured out by now that every language has tradeoffs and compromises along the way because it has to run on x86/x87/AVX512 CPUs, then I guess you will need a couple years until we can have a rational discussion about it.
Rust is good at type safety, we get it. But there is plenty of developers that are also fine without type safety. Therefore type safety is just a scale from duck to dependent types, and in Rust, type safety is usually in the way.
Yet is still directly comparable with 'i == nil' which yields true. Two different interface pointers of the same type that are both 'nil' can also be compared this way. Plus interfaces can be manipulated under reflection.
> At this point my feeling is that Go might as well stick with interfaces and not attempt to provide union types.
The language, and most particularly the runtime, is not compatible with the idea. I'm not sure why it gets brought up other than language level FOMO.
I think it gets brought up because people see places it would be helpful quite often.
A lot of the Go code I've written has:
switch x {
case ....:
case ....:
default:
panic(fmt.Sprintf("inexhaustive match, please update this switch statement for %v", x))
}
switch {
case err == nil:
case errors.Is(err, SomeErr):
// handle SomeErr
case errors.Is(err, OtherErr):
// handle OtherErr
case strings.Contains(err.Error(), "tls: unknown certificate authority")):
// handle unexported go stdlib error, XXX fix after https://github.com/golang/go/issues/35234
default:
panic(fmt.Sprintf("unhandled error type: %v: %T", err, err))
}
I could have my own functions return up a non `error`-interface type, not implement the `error` interface intentionally to avoid the nil-typed-interface problem 'err' is otherwise prone to, and then I could actually know when I compiled my codebase that my error handling is good, and know what needs to be updated after adding new error paths.
What I am yet to find in any programming language is a reliable way to put my cursor somewhere in the code base, and know exactly all the possible errors which can happen, where they could come from, and whether I've handled them (or might be interested in handling them).
Fault-injection is also consistently a sore point - i.e. in Go if you're doing an os.Open(somefile) call - it'd be really nice if writing code which tests the failure path was just easier - i.e. most of the time I'm saying "this Open on this line, just return an error for this test case so I can make sure it does what I want" - without bogging myself down in dependency injection or call monitoring (i.e. realistically I just want to say "os.Open in file somefile.go, line 143 - for this test please just return err"). Yes something like this can be done, but can we really not make this easier? Write tools which make this easier?
Ideally we'd have unions which gets rid of the bike shedding that comes with sums [1].
(Zig is similar but doesn't support different types of payloads on the different error values).
In your case you can either model it with a sum type or wrap each variant in its own type and take the union of them.
Example:
enum Result {
Success(String),
Error(String),
}
struct Success(String);
struct Error(String);
type Result = Success | Error;Zig's error types and error unions seem like a decent approach to achieve this.
You can do this in Scala with an IO type e.g ZIO.
All functions have a return type of IO[Success, Error] and it will always return one or the other.
However, in the near future, the compiler itself should be able to handle all exceptions statically:
- https://docs.scala-lang.org/scala3/reference/experimental/ca...
- https://docs.scala-lang.org/scala3/reference/experimental/cc...
The LSP doesn't show where the exceptions are raised from, but that'd be cool. Even just having a link to pull up a search for a given exception would be handy.
If only they could solve this syntactically...
For other programmers, it's missing critical features.
For me, it is very important to be able to model my domain in the type system. You can't do that very well (compared to other languages) with Go's type system.
I would choose Go over any dynamically typed language. I wouldn't pick it over most statically typed languages.
The emphasis on simplicity is a laudable goal. The problem is they picked the wrong definition of "simple"
Go’s type system is really not much better than Java, if it’s not worse.
The reason this stuff comes up is because many folks are forced to use a language for commercial reasons due to job. It's not as simple as just fscking off to use something else you desire. Good luck selling that rewrite.
Sometimes autocorrect is serendipitous.
Suppose we talk about tagged unions specifically, would it be conceivable to just nominate one variant as the default variant?
Hence, Option[T]’s zero value could be None, and Result[T, E]’s zero value could be Ok(T’s zero value).
(I have no idea if this would work or not. I haven’t used Go for a dozen or so years, but once knew it pretty decently. I work extensively in Rust. But I am curious about this, because I want every language to have sum types, they’re just such a great thing. Always miss ’em when working in JavaScript or most other languages I ever touch.)
My experience with Go is just as recent as yours, and I'm happy to eat shit if I'm completely off base here, but my interpretation is that a tagged union with two instances like
Some a | None
Compared to Haskell, where for
Maybe a = Just a | Nothing
It's not that strange. That's the normal way to model these things in OO languages, and while it's obviously not ideal, it's not completely awful either.
Literally the only two features I’ve ever wanted in go is a way to express optional return values without pointers, and a way to be able to write a set of enumerable values in a sane way. The inability to express both in Go is quite frankly ridiculous.
I use go extensively. I’ve written numerous tools and deployed lots of things to production with it. Both of these problems are such a sore point for me. So many go libraries have either ridiculous workarounds with foot guns due to these two missing features that it hurts to use most of them.
> The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt
How would you phrase this? Thinking programmers are too stupid to understand languages with modern features is just the plain literal meaning of the quote, no?
these same lowly colleagues were also incapable of wiping their asses at one point in their life but over the years and skidmarks, they all mastered that beautiful language and now make heated bidet money.
So a more charitable interpretation would be that most of his colleagues are in their 20s and just out of school with a CS degree. He doesn't think they can (productively) use the languages the cutting-edge PL researchers generate.
But trying to get new grads to use Haskell on the scale of a 10 million line code base, and have them not make a mess of it? That's the kind of thing he didn't trust the new grads with.
Even if you believe that anyone not capable of writing Rust or Haskell is stupid, it doesn’t mean that Rob Pike thinks that these people are stupid.
I think a really important insight from ~50 years of PLs is that recent language features (say: lifetimes, or dependent types) do not always correlate with practical adoption, security guarantees, low cognitive overhead, teachability, readability, fast compilation, mechanical sympathy, and other such goals.
You can totally argue that Go should have been designed differently, but it's much harsher and untrue to say the designers ignored the ideas you have in mind.
Generics was a big thing because it impacted how a lot of code was written. It doesn't mean people don't want other things in the language that is hell bent on ignoring any practical advances in languages of the past 50 years.
What you propose is to basically stop improving the language because people complain.
Sum types and product types are fundamental to a type system, the same way that addition and multiplication are fundamental to arithmetic.
You wouldn't design a language with only multiplication and not addition. You wouldn't design boolean operations with only `&&` and not `||`. You wouldn't design bitwise operations with only `&` and not `^`. You wouldn't design set operations with only `∩` and not `∪`.
The alternative to "ignore" here is "ignorant", so it seems a nicer intention that they were aware of the fundamentals and chose not to use them.
After one week, i started calling Go: C+-. It felt like a superset of C with a lot of helpful tools, that kneecaped you each time you want to do something it's not meant to, like using memcpy. Why feel so much like C and not give you its most powerfull tool? (i was becoming pretty good with C memory management, pointer algorithms, and gcc at the time too, and not having those tools available to code/debug probably gave me a bad first impression).
But it did its job pretty well in the end.
Regardless of their intention, it turned out to be a competitor to Java, not C.
value := make(option[int, bool], true)
b := value.(bool) // panic if not a bool
switch value.(type) {
case int:
case bool:
}
You can do roughly what you're asking today with interfaces.
pub struct PrimeNumber(u64)
impl PrimeNumber {
pub fn new(n: u64) -> Result<PrimeNumber, PrimeError> { ... }
}
Rust has tons of these, e.g. NonZeroU64 is guaranteed to be >=1. If you are somehow holding one, you don't have to check.
You can e.g. make a `struct Username(String)` and all paths to creating one enforce it to match a grammar; you will never encounter a `Username` that's the empty string, or contains non-printing characters, etc.
If you think of solving that by having a way for PrimeNumber to say "my zero value is actually 2", well, a) that's really weird and will just cause different bugs and b) that only solves the very simplest case.
Example (with proposed syntax):
type Username struct {
value string
}
func NewUsername(value string) option[Username, UsernameError] {
if value == "" {
return make(option[Username, UsernameError], NewUsernameError(..))
}
return make(option[Username, UsernameError], Username{value: value})
}
u := NewUsername("")
switch u.(type) {
case Username:
// ...
case UsernameError:
// ..
}
If you don't have that, you're not really improving on `value, ok := foo()`.
As I said, with the proposal above, you cannot manufacture a default zero value from a union if you specify in the language's spec that trying to retrieve a value from a non-initialized ("not active") subtype panics (i.e. the union was constructed with a different underlying type)
asUsername := myUnion.(UserName) // panics if myUnion doesn't hold UserName internally
// if you want to retrieve the value, you type-switch:
switch myUnion.(type) {
case UserName:
// if the underlying type is UserName, it always returns the value as it was passed into the union's constructor by the developer, it doesn't matter if it was "initialized" or the developer passed some default zero'd value; it's not the union's concern at all
}
Whether Go must allow non-zero types in general is a totally different topic.
Really I've never been convinced that default values are the huge security or safety issues they're made out to be. Perhaps there's a bunch of CVEs due to it but all the CVE's I've ever read are due to memory bounds issues, use after free, or not zeroing out data properly and leaking info. Also I'm talking about languages where all values are properly initialized to a known zero state. Not C's worst case where a value could be full of random data.
Sure there's some scenarios, say encryption keys, where default values could be an issue. However a programmer can just as easily set an encryption key to `default()` in Rust and end up with the same issue. It may be "explicit" but programmers get in habits of using things like `default()` and so it's still as likely to happen as it would with non-explicit defaults IMHO.
Requiring non-zero values isn't worth the mental overhead in the other 99.9% of cases where zero-values can simplify a lot of coding to say declare a stack array of zero-valued structs or unions, etc. In those cases you can make the zero-value of a union type where the zero value is `UnInitializedBadEncryptionKey`.
> As noted in a comment by loreb on my entry on how union types would be complicated, these 'union' or 'sum' types in Go also run into issues with their zero value, and as Ian Lance Taylor's issue comment says, zero values are built quite deeply into Go.
What are the actual issues though with zero values in Go when used with union types?
> You can define semantics for union types that allow zero values, but I don't think they're really particularly useful for anything except cramming some data structures into a few more bytes in a somewhat opaque way, and I'm not sure that's something Go should be caring about.
Why not? Generally the zero-value for a union type is just the first option, which can be useful or not. You can wrap it in an option type to ensure it's been initialized too.
If Go's semantics mean it's always a nullable pointer, then that still doesn't make union types useless. There's still value in knowing that "if there's a value, it's one of these Y options".
Though it does make sense that in Go that type assertions in Go could be used at the compiler level to say that "this variable is one of Y interfaces".
I don't think that "zero-values are inherently unsafe" is a Rust philosophy? I think it's more about some sense of control and/or design - Rust shouldn't force types to have a zero value if a zero value doesn't make sense (e.g., references, NonNull<T>), and Rust shouldn't force your type to have zero as its default value if another default would be more appropriate.
> However a programmer can just as easily set an encryption key to `default()` in Rust and end up with the same issue. It may be "explicit" but programmers get in habits of using things like `default()` and so it's still as likely to happen as it would with non-explicit defaults IMHO.
default() would only be usable if the encryption key type implements Default and I think I'd be a bit on the skeptical side with respect to how likely Default would be "accidentally" implemented on it. Putting aside questions about how Default on an encryption key made it into the codebase in the first place, IIRC array types for representing crypto keys are not that uncommon and I don't think Default is implemented for arrays (by default?), so #[derive(Default)] wouldn't even compile and I'm not sure impl Default would be common enough to be something that's done out of habit.
Specific example aside, I'm not Default is one of those traits that are habitually added to structs in the first place?
> Requiring non-zero values isn't worth the mental overhead in the other 99.9% of cases where zero-values can simplify a lot of coding to say declare a stack array of zero-valued structs or unions, etc.
Would it really be that much simpler? For example, in Rust for Copy types:
[DefaultStruct::default(); SIZE]
[NonDefaultStruct { value: defaultValue }; SIZE] // other options available as well
> In those cases you can make the zero-value of a union type where the zero value is `UnInitializedBadEncryptionKey`.
The disadvantage to that approach is that now everything that deals with that union type needs to know about/deal with uninitialized values even if such values are only ever transient during initial construction and are supposed to be impossible otherwise. It's arguably cleaner to make such a state unrepresentable at all and use a different mechanism to represent being uninitialized. For example, Rust again:
let uninit = [const { MaybeUninit<NonDefaultStruct>::uninit() }; SIZE];
// initialize as appropriate
let init = unsafe { std::mem::transmute<_, [NonDefaultStruct; SIZE]>(uninit); }
Structures are often on stack, but will get boxed if their reference escapes analysis to not outlive stack slot
The point is you don't offer any default if there is no safe default, and so it's not so easy. Yes, a determined programmer can use any API badly. But nudging them in the right direction helps move the needle.
It reminds of PHP's spirit of moving forward despite errors as far as it can go. It looks like it's working, but you can never really trust such a program much.
Races with the GC, if you try to be smart and have unboxed options.
Interfaces are similarly boxed to avoid such races, and make the rest of the language faster.
> If Go's semantics mean it's always a nullable pointer, then that still doesn't make union types useless. There's still value in knowing that "if there's a value, it's one of these Y options".
At that point, what's the benefit over a (boxed) interface and existing type assertions/switch?
Besides, if the options are your own types, you can also add an unexported method to the interface to seal it. And then, if you go this route, you can also have exhaustiveness checks with existing tooling.