Generating code
blog.golang.org
blog.golang.org
This looks much like my team's `make update` rule (that merely runs ./mk/update.sh). Well, except now the dependencies on external tools are scattered throughout your project. Oh and build failures wouldn't even tell you that you need to run go generate.
What a mis-step. Then again, maybe I'm somehow mistaking a build system with a compiler[0]?
If you need a more elaborate build system, use one.
I thought I was:
https://golang.org/doc/articles/go_command.html
> An explicit goal for Go from the beginning was to be able to build Go code using only the information found in the source itself, not needing to write a makefile or one of the many modern replacements for makefiles. If Go needed a configuration file to explain how to build your program, then Go would have failed.
No, not at all. Not even close.
Your go generate comment takes the form of
//go:generate someCommand arg1 arg2
This is a terrible mess if you want to write a cross-platform generator and it's an even bigger mess if your generator isn't something present on every platform already because otherwise generate will just fail with "command not found".
But you know what, that doesn't matter either, it's true for shell scripts and it's true for makefiles as well. Go generate changes nothing in that respect, however, it does improve portability significantly in other ways. For example, in the Go tree we replaced shell scripts with go generate and perl scripts with go code called by go generate. This is important because Windows and Plan 9 don't have bourne shells, Plan 9 doesn't have perl at all and in Windows perl has to be installed. Note that there are no makefiles, which don't exist natively on Windows and Plan 9 either.
I definitely agree with you on extracting rules into separate scripts, but I've yet to see a make-based build system that embraces this idea.
[1]: https://github.com/gvalkov/makefile-shell-backslashThis applies the "project info in the source" and "make stuff doable through the go tool" ideas to codegen. I don't think it's life-changing (I hope it's not too life-changing, i.e., that people don't go too crazy building hard-to-maintain Go-generating hacks) but it seems consistent.
The article only gives one reason why generated code should be checked in ("if only for the reason that the program it invokes might not be available on the target machine") but I suspect there might be other unstated reasons: it's useful to have a permanent record of generated code in source control for auditing and debugging, and it ensures that package authors can't make downstream builds slower by choosing a slow build tool.
In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing.
I've definitely found diffing the generated code useful for confirming refactorings of the generation code don't change anything unexpected - and I imagine those doing my code reviews do as well. I might not make much use of the actual VCS history, but committing to VCS gives me sane diffs to look at for minimal effort.
> In the open source world, your dependencies are maintained by people in other organizations. Avoiding dependencies on other teams' crufty build files is a good thing.
Agreed (although sometimes the build files are good.) I'm pretty sure that moving crufty build rules into such a primitive build system isn't help on that front however. Even as a Microsoft coolaid drinking, Visual Studio using, C# loving windows dev... I think I'd much rather hack up or rewrite someone's Makefile than go diving through the source to find all the similar-but-not-quite-the-same build rules scattered throughout the codebase.
I'm not very familiar with go but I think the motivation might be inspired by Python's "one way to do it" maxim. https://wiki.python.org/moin/TOOWTDI
This seems to be the philosophy adopted by Go as well (e.g. the gofmt tool, the integrated build system, only one loop construct, slices, etc.).
Personally, I think this philosophy is extremely beneficial in making code more readable and I would say this feature is consistent with it.
The motivation is pretty transparent: since the Go team is categorically refusing to implement generics, they are trying to find workarounds to avoid having developers copy/paste code to simulate genericity.
Their solution is to have a tool generate that code instead of developers doing the copy/paste.
However I'd admit that I don't know windows and real motivations behind it.
Go generate is probably the most half-assed thing the go team has done yet.
Now, I know the above line is harsh, but I don't say that without proof. Run the following and bear witness to its terrible implementation.
curl http://pastebin.com/raw.php?i=fhu14iwk > main.go
cat main.go
go run main.go
go generate -v main.go
go generate -help
go generate -x -v -run ".*(echo|one).*"
If you don't feel like looking at the above, the implementation of go generate is a hack of substring matching, not language parsing, and it also doesn't implement, in its production release, the "-run" flag documented in "-help".
Maybe I can agree that // prefix is a bad choice, as coming from C world I would prefer #
Define might be. But actual directives are not. In go, the directives are.
I would think it would be way simpler to only allow those special comments in the input file for "go generate".
If I were snarky, I would add that, if this gets used a lot, it might be handy to add dependency information to that file, so that "go generate" can skip repeated work. Once it does that, people can call it every time they compile their code, gaining convenience without much loss of time.
I also wonder what the distribution of code will look like. The goal seems to be to shield ordinary programmers from code generation. That is somewhat laudable (I would like to see a 'layered' language where upper layers are highly similar, but simpler to use and a bit less powerful, but haven't seen a good one yet), but at odds with the GPL. I think the GPL will require you to distribute the master code.
When used as an alternative for generics, distributing only the generated code also will lead to diverging of the various generated versions. People will fork and then modify only eight of those ten generated files.
I can't read the wikipedia these days, there aren't enough opinions.
But man, I don't know...having to manually run a generate command when eg changing string constants seems pretty weird. And if you change the constants without rerunning go generate (manually!) you may get nonsense values. It seems pretty creaky!
Maybe it will end up working out well in practice. (As Go tends to do.)
And I do appreciate the minimalistic nature of the approach...
I expect we'll start to see people do a lot of generics-type stuff with this -- e.g., add an go:generate annotation to get a customized Foo collection or whatever.
It'll be interesting to see how it shakes out.
I'm would wager that generics were a driving force behind this. They don't want to add them to the language (fair), but are willing to offer a way for us to add them at precompile time.
The driving forces were yacc, protocol buffers, enum strings, unicode tables, encoding/decoding arrays, time zone data. That's what they use it for.
That said, if I'm reading that code right, if I, say, decide to reorder a constant definition (say swap Aspirin and Ibuprofen), the Go code will still compile, but the string constants will be silently wrong until I realize I need to rerun go generate. No?
It's a problem in general: if I modify code that requires rerunning go generate, and I forget or don't realize I need to do it, I'm going to get at best a confusing compiler error, and at worst silently wrong code... Right?
Again, maybe this all works fine in practice, but to just brush off these concerns seems a bit glib...
For instance in this case, the very existence of the code generation libraries contradicts the reflection package.
Compile-time function execution would have been so much more consistent it's horrifying :
Example vaguely similar to Go's example:
http://forum.dlang.org/thread/lgunf8$14rp$1@digitalmars.com#...
http://stackoverflow.com/questions/18552454/using-ctfe-to-ge...
But like every other decision in Go, you can either assume the Go developers actually believe the arguments they're putting up ... or you can ask yourself "would the Go compiler/toolchain become easier or harder to implement ?" And use that argument as almost the sole argument for any decision. One of these ways of thinking explains every decision very consistently, the other leaves one listing the exceptions.
And the right solution, mixins + CTFE is legendary hard to implement. As in, someone who's written 3 commercial C++ compilers from scratch has serious trouble getting it to work.
But it's such a joy to use.
That's not accurate. There are a lot of things about Go that are very difficult to implement. The concurrency system and garbage collector are obvious examples. Some less obvious ones are discussed here: http://talks.golang.org/2014/compiling.slide
Conclusion
Go is simpler to compile than most languages
There are still complexities for the compiler
Most complexities stem from making Go easier to write
//go:generate stringer -type Pill
const (
Placebo Pill = iota
Aspirin
)
Or worse: what if there's a tool called "stringer" on your device that isn't what is on the developer's machine ?
Even worse : what if it's subtly different ?
[ps. yet thank you big time for being able to program in golang]
It's not part of the language, it's not in the spec, it's not part of the compiler.
In a typical Common Lisp project, Lisp macros generate the code at compile time. Since a full Common Lisp is available at compile time, this looks similar to runtime, but it isn't. For that the macro itself and the code it needs to run needs to be available during compilation.
During development the compile-time and runtime code often run inside the same Lisp system. But that's a certain style of development and not necessary. It's also possible that the generated code will form an application, which then runs outside of the compile environment.
To take a typical data structure declaration and to generate compile-time code for it is quite typical. What Lisp typically not do is to dump the generate code as source to a file, but to feed the generated code to the compiler and then create compiled code.
(Why does every language now have to come with its very own build/packaging environment, with its very own directory structure conventions? Go has one, Rust has a different one, Java has several, and a new one for Python was discussed here yesterday.)
That said, go tools are canonical but not mandatory. If switching to this doesn't work for the protobuf foks, it's simple enough not to switch.
Committing cpp output makes no sense because cpp output is specific to the machine which ran cpp.
tl;dr Go has its own version of the make tool set that doesn't allow dependencies outside of Go source files, and doesn't rebuild dependencies on file updates.
Or at all. 'go generate' does not build go source, unless your comment is "//go:generate go build" which would be, to say the least, pretty weird.
Think of 'go generate' as a codified 'gen.sh'.
The Go philosophy is that if you truly need a make file or other such system, use it. It's always been that way, actually; if you poke hard enough you can still find old Go tutorials which require you to use make (or some other build tool). But if you need it for just this simple little thing, you shouldn't have to pull in make, just as the developing Go philosophy is that it is silly to pull in entire packages for one little function. The purpose of "go generate" is not to replace make; it is to close out another 80% of the uses of make (my made-up number) by offering a simple, high bang-for-the-buck addition that can be depended on in the base language.
Some projects will still need make. This expands the set that don't. There's no interest in trying to make this grow to include all things that use make. It fits the pragmatic philosophy of Go.