Calculating Go type sets is harder than you think
blog.merovius.de
blog.merovius.de
> ...
> This is especially important for environments like the Go playground, which regularly compiles untrusted code.
To be clear here, the tradeoff being made is that it is more important that the compiler is fast than that it is able to tell programmers about clear bugs in their code.
I see the justification of the playground as unconvincing too. You can run 'for {}' in the playground, and you'll get "timeout running program". They already have solved the problem for the playground because obviously at runtime a go program can take infinite time, and it doesn't seem like it's any harder to also constrain compilation in the same way.
The actual cost that the go compiler being fast has is that other parts of go are slow. In other languages, a macro will execute as part of compilation, and incremental compilation will work with it. In go, if you want a macro, you write a separate go program that generates code, write a "//go:generate" comment, and then have no way of knowing when to run it, no way of having incremental compilation with it, and it invariably ends up being slower than the compiler expanding it itself would have been.
Now, if you want to know if you have an impossible type set, you'll run "go vet" (which likely doesn't do this validation, I haven't checked) or one of the various linting tools, it'll have to re-parse and re-compile all the code, it won't cache, and it'll ultimately give you the same answer the compiler could have given you.
I suppose this is superior in one way: when performing such generation and validation, you can do it against only your own code, rather than against third party code you import, while if it were part of the compiler directly it would typically end up running against all compilation units, even third party ones....
In practice though, even for "slow to compile" languages, I find that I spend more time finding and fixing bugs than I do waiting for compilation, especially with incremental compilation. On the other hand, with go, I spend less time total compiling code, more time total fixing bugs, and more time total running golangci-lint and various other bits of code-generation.
I spend exactly 0 time worrying that some adversarial third-party dependency I import will make my compile-times exponential, both in go where it can't happen by design, but also in other languages where it could happen, but you know, doesn't.
You run it every time you do a commit (at least), and then you check in the changes.
Also, you probably run it during development by using a Makefile that will automatically run it whenever the input changes.
This means that downstream packages never have to run it. They don't even have to have the code generation tool installed. This means a package maintainer can use any code generation tool they like.
It was probably done that way due to experience with Bazel, where doing a build means you need to build and run code generators written in multiple programming languages. It does indeed slow things down.
In this case, you know to run "go generate" because the feature you just added doesn't work. (Though I tend to add a CI rule to make sure that the generator in CI produces the file that is checked into the repo. Because, yeah, you can forget, and it's nice for the computer to double check that.)
Not necessarily; there is plenty of checked in generated Go code at Google (as well as other PLs, but probably not as much), and other projects (e.g. Entgo) seem to encourage checking in generated Go code as well.
For example, suppose I'm writing some mocked unit tests in Go and want to use mockgen to generate some mocks on an interface...
//go:generate mockgen (N.B. This generation is done by Bazel)
type Object interface {
Thing() string
}
mockThing := NewMockObject(ctrl)
mockThing.EXPECT().Thing().Return("Foo")
The code will compile and run successfully, but the developer experience is terrible because you lose the ability to use your IDE's code lens, syntax checks, autocompletion. Everything that's generated shows a red squiggly this-doesnt-exist line.
Maybe I'm just fundamentally misunderstanding how Bazel is supposed to be used. My setup is pretty typical. I develop in VS Code, and build using Bazel, but I do not allow Bazel to automate code generation because the generated sources do not appear in my local environment - just in the bazel sandbox.
Maybe someone can suggest a better approach?
Here's the bug: https://github.com/bazelbuild/rules_go/issues/512
I basically wouldn't use Bazel with Go. Go already has a build system, Bazel is best for languages that don't ship a build system, like C++.
I think I agree that Bazel is poor for Go code, sadly it's what our SREs decided to use for our monorepo.
My rust experience denies this.
The actual tradeoff being made here is more like "it's better to give a good error message in 1s than a great error message in 10s." That doesn't seem obviously wrong to me.
1. Macros are just one use case replaced by go:generate step.
2. go:generated files are usually not the most moving parts while developing (except of course while you write the generator or its input. In fact go:embed has allowed to move some generated files to compile time and have not seen the need to replace go:generate with go:embed for those. (Does anyone has a compile benchmark?)
2. The Go compiler already uses caches and we can expect more and more caching will happen in future releases.
So I'm really not concerned by the compile time of files produced by go:generate instead of macros.
I'm not sure that's entirely true. Or it seems at least misleading.
"Fast" here means "guaranteed to finish before the heat death of the universe", in the worst case.
And as I'm trying to explain in the post, it is easier to explain clear bugs to the programmer, if you don't have to rely on solving NP-complete problems. NP-complete problems are notoriously hard to report errors for, which the user understands.
I've never seen a concrete type in an interface before, so I decided to try it:
package main
type A interface {
int
M()
}
func f(x A) { // line 8
x.M()
}
./main.go:8:10: interface contains type constraints
Since this is in the context of generics, I think the author might have been thinking about embedding types like comparable, which are interfaces (comparable is interestingly defined as `type comparable interface { comparable }`). You can, of course, put interfaces in interfaces in general (see io.ReadCloser and friends).
For the comparable case:
type A interface {
comparable
M()
}
func p(x A) {
x.M()
}
./main.go:8:10: interface is (or embeds) comparable
I don't know what the point of this comment is, since 99% of the article is not about this code block, but I guess I found it interesting.
> interface is (or embeds) comparable
Currently, not all interface types may be used as value types. Such types include comparable.
It is very possible that all interface types could be used as value types in future Go versions, and comparable might be able to be used as value types as early as Go 1.20.
You can put concrete types in interface definitions as of 1.18. See the spec: https://go.dev/ref/spec#Interface_types
https://go.dev/play/p/U25AzZndz-7
If you wanted to argue that you can't use it everywhere you could use an interface before, that'd be all well and good—but your assertion
> you can't actually put concrete types in interface{} definitions
is just incorrect, in a way that lead you to misunderstand the article.
You can't use it as a type, but you can use it as a constraint: https://go.dev/play/p/LnmudHIjAoQ
Of course, you can then not actually instantiate that function (as the type set is empty). But using concrete types in interfaces is very possible. And it's indeed why the |-operator was introduced. It allows you to use operators: https://go.dev/play/p/I3eMDBwIcHA
Is it pride that let's "industry" languages ignore decades of PL theory and research? The "design by committee" problem? Or something else entirely?
And it's fine if the error messages are helpful, but this added implicit complexity to the language rightly gives folks the heebie-jeebies (se the recent comments for Rust's GAT patches)
NP-hardness only shows the worst case is really difficult. It does not mean that an approach is not useful. For example, Rust's exhaustive pattern matching is also technically NP-hard: https://niedzejkob.p4.team/rust-np/
But these "Go's ignoring decades of PL research!" comments are, at best, tiresome even when somewhat true. In this case it's the outright opposite.
Its not academia, the grade is getting fast funding or revenue because any language is fine and any performance is good enough because the bottlenecks are somewhere else or irrelevant
So, in fact, industry didn't ignore PL theory and research at all.
Even C++ arguably falls into that category to an extent. Its template system definitely could have been designed better, and using Haskell-like type signatures like Rust does is arguably a better approach… but Rust trait signatures tend to be more complex and less elegant than Haskell ones, because the simple and elegant ones have runtime overhead that Rust, like C++, isn’t willing to accept. (For example, see functions being a trait as opposed to a type.) And even Rust struggles to match C++ in metaprogramming expressivity, which also helps C++ reduce runtime overhead. Upcoming Rust features like `const fn` and specialization will help with that, but both have been ‘upcoming’ for many years…
Instead of belly aching about everyone who did it wrong, why not explain what is wrong and why you think so.
I mean, I think think Rust didn't do it perfect because I often want to use Box<dyn MyTrait> which is a hassle because I need to mark every function ?Sized if I want to pass a reference to my `x: &T...where T: MyTrait + ?Sized` and I don't enjoy that.
Sometimes, reality bites.
As in all of engineering, most things are a tradeoff.
The other problem is incentive and design approach. The academic approach to PL design is to carefully develop the type system and understand the way the rules interact, proving things like soundness, decidability, etc. the industry approach to software development is MVP. These two approaches are incompatible and lead to different language designs, because you can't slap a carefully designed system on top of a recklessly developed MVP that has been bumped incrementally over a long time. At least not without major time consuming rewrites.
FWIW I've used Rust for a bit and I disagree that they "got it right". I'm not saying Rust is bad, I like the language. But it's not as if their generics implementation does not have issues. It's just that they are different issues.
Checking type sets is a work for a linter. But the developer should have caught the problem with a simple unit test.
I'm glad that the specification relieves the compiler writers from that the non-empty check task.
Linting can happen asynchronously in CI.