Golang: Don't be afraid of makefiles | Hacker News Reader

Golang: Don't be afraid of makefiles

sohlich.github.io

177 pointssohlich154 comments

Two things I try to get people to do with Makefiles:

1. Use the := syntax

:= means that a variable is immediately expanded; = means it's delayed until the variable is used which on a large Makefile has a speed penalty. If you know that a variable is fully defined (i.e. all the $(...) references in its value are fixed) when the variable is being defined then use :=. That's the case here where the Makefile starts:

    GOCMD=go
    GOBUILD=$(GOCMD) build
    GOCLEAN=$(GOCMD) clean
    GOTEST=$(GOCMD) test
    GOGET=$(GOCMD) get
    BINARY_NAME=mybinary
    BINARY_UNIX=$(BINARY_NAME)_unix

which can be written

    GOCMD   := go
    GOBUILD := $(GOCMD) build
    GOCLEAN := $(GOCMD) clean
    GOTEST  := $(GOCMD) test
    GOGET   := $(GOCMD) get

    BINARY_NAME := mybinary
    BINARY_UNIX := $(BINARY_NAME)_unix

2. If you have a one line recipe then use the target: preq ; command syntax to avoid tabs. For example,

    build: 
        $(GOBUILD) -o $(BINARY_NAME) -v

is the same as

    build: ; $(GOBUILD) -o $(BINARY_NAME) -v

While it is not so important in simple cases, '=' allows to not depend on the order of definitions, so you can group variables in logical order, not as statements. You can look at := as variable definition and = as semi-function definition. Penalty of delayed-but-many variable expansion that doesn't contain shell code is questionable. Also, with := half of them may not be used by 'maked' rule at all, actually losing performance.

There is also ?= which only sets a variable that was not set before or via command line or environment (make inherits current environment variables iirc).

> and = as semi-function definition.

Variable binding.

Well, = doesn't really behave like traditional variable binding, since you can do stuff like:

    build = cc -c -o $@ $<
    file1.o: file1.c
        $(build)
    file2.o: file2.c
        $(build)

(This is meant to illustrate the difference, this isn't a good makefile.)

To reinforce the notion that variables are really functions, you can even supply arguments when you call them:

    # reverse is a function
    reverse = $(2) $(1)

    # eventually expands to 2 1
    a = $(call reverse,1,2)

Please don't rely on environment variables. It plays havoc with reproducibility.

I'm usually adding 'vars' target to my makefiles that simply echoes them to see if anything is wrong or simply to see defaults. There is also command-line option to do that (and of course -n), but it prints too much to comprehend.

That's good advice, but in this case I'm wondering, why use variables at all? The makefile is only being used as a collection of aliases. They could all be inlined without losing anything.

I agree. This Makefile would be clearer with something like:

    BINARY_NAME := mybinary
    BINARY_UNIX := $(BINARY_NAME)_unix
    
    .PHONY: all test build clean run deps
    all: test build
    build: ; go build -o $(BINARY_NAME) -v
    test:  ; go test -v ./...
    clean: ; go clean && rm -f $(BINARY_NAME) $(BINARY_UNIX)
    run:   ; go run $(BINARY_NAME) -v ./...

    deps: DEPS := $(addprefix github.com/markbates/,goth pop)
    deps:  ; go get $(DEPS)

    build-linux: ; CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o $(BINARY_UNIX) -v

    docker-build: ; docker run --rm -it -v "$(GOPATH)":/go -w /go/src/bitbucket.org/rsohlich/makepost golang:latest go build -o "$(BINARY_UNIX)" -v

Caveat. I haven't tested that, but something close to that would be much clearer.

Well, the idea is that maybe the command 'go' is different on different build systems/targets and if you need to change it to something else (go-x86, or something like that) then it's going to be more difficult the way you wrote things.

Indeed, those variables are very handy for cross compiling. If you don't do anything else, at least define and use a $(CROSS) prefix. (Or, don't and wait for some cross compiling distro to approach you and to offer a patch that introduces such a variable.)

Sure, that's what variables are for.

But in a small makefile, it's a simple search-and-replace to move it out to a variable. Adding indirection slightly degrades readability, so better to wait until it's first needed.

Very true.

Should `all` also be in `.PHONY` or is it a special case?

I just forgot it.

Having a bunch of variables is more useful once you get into Auto{make,conf}, where programs like the compiler can be auto-discovered and can be overridden by the user at configure-time.

> = means it's delayed until the variable is used which on a large Makefile has a speed penalty

Do you mean it's expanded every time it is used, vs once? If not, it's not clear to me why it would be slower.

It's deferred interpolation, which means every time.

Ok, so very painful at scale.

Shouldn't it be insignificant, even at scale?

Heck, JS spawns functions at scale for everything, and we're worrying about mere interpolation?

In this case: doesn't matter so much. Small text interpolations should be fast. In the more general case, a variable might be bound to a shell command, a huge list of files, or built-in Make functions doing a lot of work. And you might be using that variable in a thousand places.

It makes sense not to repeat work that doesn't need repeated. Even more so if it could significantly slow down the build.

The make variable interpolator may not be as optimized as the javascript function spawner.

Yeah, but it still shouldn't matter, because there's not much to optimize: string substitution is inherently fast.

But it's what your sibling comment said that I missed: it might not just be string interpolation, it could bound to a shell command's result etc -- which would then be executed at every occurrence.

I wouldn't be so sure about that. make has been around for several decades now, and it affects the build times of a huge number of projects. Most common versions of make have been very tightly optimized, in ways that you might not expect.

Makefiles date from an era where a 200MHz machine was fast and 32MB of memory was a lot. It's not really as painful as you think.

There's another good use for Makefiles with go.

Gopath is one of the dumbest ideas I've seen. Multiple large projects under the same directory? Who thought that was a good idea. (and yes, I know you can create subdirectories, and mix all your packages together to create a giant mess.)

So I separate the projects into their own directories.. and use a Makefile to set gopath and run the go commands.

Still trying to figure out why golang tried to be innovative here.. What couldn't they use cwd like every other compiler I've ever seen?

Yeah, that is what turned me off of Golang. It was kind of a "straw that broke the camel's back" thing, though. Go is just an EXTREMELY opinionated language, and if you don't like that then you won't like it. The Best Way to do everything is decided for you ahead of time, which irks me because it isn't always true. But I can see the frustration it grew from when I use a language like Ruby, which has a half-dozen nearly-identical ways to accomplish most simple tasks.

Anyways, you could muck around with symlinks and crap, but come on. If I can't simply mkdir/vim/compile/run a 'hello world' program, there's probably something wrong with the language.

> If I can't simply mkdir/vim/compile/run a 'hello world' program, there's probably something wrong with the language.

Like this?

`cat $HELLO_WORLD_PROG > /tmp/hello.go && go run /tmp/hello.go`

Yeah, like that!

But you know what I mean; multiple scattered workspaces, project folders on removable storage, etc. That sort of thing should be painless.

I've definitely had some other minor pain points with GOPATH, and I hate when veteran users of another language trivialize my pain points with their language, but I think all of the use cases you've described are all well within GOPATH's wheelhouse:

export GOPATH=$HOME/go:/media/USB-DISK:/foo/bar/baz

Now Go can find any projects in any of those workspaces. The only caveat is the projects have to be in a workspace, which is a directory with a src folder containing your project. I agree that this is a minor nuisance and I would design it differently, but Go's tooling is still far easier than pretty much anyone else's (it's actually what brought me to Go).

> but Go's tooling is still far easier than pretty much anyone else's (it's actually what brought me to Go).

Really?

Maybe that's true if you compare it to PIP (Python), Gems (Ruby) or NPM (JavaScript/NodeJS).

But I think that Cargo (Rust) got it almost perfectly right, and Opam+Ocamlbuild (OCaml) is very close.

Some other languages that make Go's tooling look awesome: C, C++, Java

Cargo is comparable to Go's tooling in my mind (in terms of ease of use; I think it's better in other categories), but figuring out how to use libraries (mod vs use) and organize a multi file project is a notorious pain point with no analog in Go. This means it's harder to get started with Rust than Go (language features notwithstanding), which is pretty much what we're discussing here.

I don't really know what to say about Ocaml. I don't have much experience, but most of it has been bad. It seems like you're supposed to use Makefiles, and an Opam file. The more I learned the less I liked, and it seemed to be the consensus that the build tooling was one of the language's weak points. I will say that installing packages from Opam was a painless experience. The build tooling wasn't the only thing that drove me away, but it was right up there with the poor standard library, ecosystem fragmentation, and low-quality documentation. Which is a shame because the language features look great.

With Go, all you need to know to get started is "go get".

>poor standard library

What's wrong with containers, batteries and core?

>go get

Wow, once I've tried go the package management was utterly broken, even cabal hell looked better than this. What, no package versions? Downloading directly from git repo, are you kidding me?

> What's wrong with containers, batteries and core?

Well, between those three and the actual std lib, there are four standard libraries, which defeats the purpose of having a standard library.

> Wow, once I've tried go the package management was utterly broken, even cabal hell looked better than this. What, no package versions? Downloading directly from git repo, are you kidding me?

`go get` isn't a package manager. Go's solution to dependency management is vendoring, which has its own suite of problems, but my comment was about getting newbies off the ground, and for that `go get` is suitable.

If by opinionated you mean it ignores everything that has been learned in the last 5 decades then yeah.

It's how Google does development. There's some big wins by having a mono repo for a large organization. It's a pita though for smaller projects

I don't think Google uses the standard Go conventions.

They use their build tool Blaze which I assume is very close to the open-source version, Bazel. Here's how Go works in Bazel: https://github.com/bazelbuild/rules_go

I seem to recall hearing that was finally being fixed, but a quick search of the docs suggests not. I think it's just GOROOT (another bizarre idea) rather than GOPATH that's no longer strictly required in the latest version of Go.

I agree with you, I don't understand why GOROOT couldn't just be the directory containing the compiler, and GOPATH couldn't just be the current directory. Maybe make me put some kind of workspace file there to identify it as the project root, that'd be fine.

> I seem to recall hearing that was finally being fixed, but a quick search of the docs suggests not. I think it's just GOROOT (another bizarre idea) rather than GOPATH that's no longer strictly required in the latest version of Go.

GOPATH is set to `~/go` by default, and if your projects use vendoring (which they should), you can almost completely ignore that it exists. Packages in `./vendor/` will take precedence over anything in your GOPATH.

For the Go tools to work consistently, your code has to be inside GOPATH, which means you cannot ignore it entirely.

You can often use relative package names (e.g. "go test ./..."), but it breaks whenever you use absolute ones, and the Go toolchain is based on this assumption being true. My experience is that editor plugins (e.g. the Go plugin for VSCode) don't like this.

I use hardlinks (the "hln" third-party utility on macOS) to get around this, which means I can have all my projects under a sane directory structure under ~/Projects.

> if your projects use vendoring (which they should)

For versioning? Also:

1. I haven't seen projects on github using vendoring

2. It doesn't look like `go get` allows you to download a package directly into your vendor file

For my personal project [1] I use vendoring. But as of Go 1.8 it is still inconvenient as the main package has to be on GOPATH/GOROOT or the compiler will not find it.

To maintain it I use git-subrepo [2], a very convenient tool to embed the code into own tree.

[1] - https://github.com/ibukanov/ljdumpgo

[2] - https://github.com/ingydotnet/git-subrepo

Have a look at /coreos on GitHub -- flannel, dex and prometheus-operator all use vendoring, just looking at what's pinned. matchbox does also.

Thanks for the examples!

https://github.com/coreos/flannel/tree/master/vendor

Do most Go programmers put all their code under ~/go, or do they tend to set GOPATH on a project-by-project basis?

Quite a few current Go programmers will have something set other than `~/go` because that default only came in very recently, but you'll probably find that almost everyone uses `~/go` or `$GOPATH`.

One thing that you learn quickly when starting Go is that fighting the language's idioms (even where they're questionable) isn't going to be a good time.

Yeah, that's pretty much why I haven't learned Go. :)

Don't know about others, but since I run my projects in containers anyway, I'll usually use `direnv` on my host to set GOPATH to project directory, and then use the default GOPATH in go docker images (`/go`).

I could even do without direnv altogether, but I need it so that vim-go can report build errors on file saving.

Most put it under ~/go. It's quick and easy.

Most set the GOPATH. It's quick and easy.

I use virtualgo[0] so I can organise my work properly and not mix my own stuff up with loads of unrelated libraries

[0] https://github.com/getstream/vg

After struggling a lot, I think, I finally understood best use for GOPATH with Makefile based projects. I use it as follows:

I have ~/go for all 3rd party code that I did not develop. My personal projects live in a different directory with a different structure (including, configure.ac, Makefile.am, etc.). My projects tend to use multiple languages, so I need Makefile support.

I fix GOPATH in project specific Makefiles with ~/go as the first directory followed by project specific directory for Go code. For example, I add the following in Makefile.am of every project.

GOPATH := $(HOME)/go:$(CURDIR)/go

One cool result of this is: When I do go-get of any 3rd party package (either through command-line or as part of a rule from Makefiles), Go downloads them into ~/go directory (it picks the first directory in GOPATH or defaults to ~/go). This leaves my project specific directories free of 3rd party code -- which is what I want.

End result is, ~/go ends up with 3rd party code from across all projects and it doesn't include my project code, so can be deleted any time and re-created automatically by Makefile rules.

How do you deal with conflicting versions?

Honestly, I didn't face that issue till now.

It is also my biggest gripe with Go and it's not quite easy to workaround. I am using a Makefile approach for that: https://vincent.bernat.im/en/blog/2017-makefile-build-golang

Perhaps Dave Cheney's gb would interest you; it does not use GOPATH. https://getgb.io/

GOPATH exists to tell the go tool where to find the libraries that go get installs. You can have multiple GOPATH entries in there if you want to segregate things. It's basically PYTHONPATH with a few extra bits.

One of the reasons is that many of the developers of Go subscribe to a monorepo philosophy. GOPATH enables and encourages that practice.

You can work around the GOPATH issue by using Docker. If you build go within a Docker container, you can structure the folder however you like and it won't affect the host.

The only downside is that your text editor might stop offering some features when editing a go file, since the import paths might not point to the actual location.

I don't think you have to go that far. I think something like https://direnv.net/ should work too. Although, I do agree that having this restriction in the first place is bizarre.

The only downside being well having to use docker to work in a modern language

> Multiple large projects under the same directory

Sorry to be snarky, but everything, eventually, is under one directory. "/".

There seems to be two kinds of folks working with Go: those who want to do it their way, and those who want to do it the Go way. I'm in the latter group.

> ... I separate the projects into their own directories

So do I, but I just don't get to pick that directory. Everything is in $GOPATH. $GOPATH/src/github.com/:org/:package is my directory for any given project. That project has a /vendor directory with its dependencies. Everything is contained within my project's directory. It is in source control. Everything just works with the Go tooling.

> ... use a Makefile to set gopath ...

If you don't fight the system, you don't need to set GOPATH per project. I totally get there is something that maybe I just don't understand or am totally missing, but in _years_ of working with Go on many dozens of applications, I've never had to alter my GOPATH (aside from the one application that a team at our company did where they put their source control at the $GOPATH/src directory, which was a pain in the butt and is not the way to do things).

> Still trying to figure out why golang tried to be innovative here...

Yeah, it was weird for me at first. I accepted it as the way of things and my life got better.

And to harken more back to the heart of the thread: makefiles. While I just don't understand the need for separate GOPATHs, I can see some value for complicated build or test commands. For the most part, `go test -race` and `go build` is what I need 90% of the time. Mostly, it is the build system that needs the equivalent of `make test` and `make build` to ensure flags are set and such.

I believe the idea behind it is that you should only deploy one app per container. Perhaps developing locally with Kubernetes would be a simpler path.

Whenever someone says "X is one of the dumbest ideas" there are two possibilities:

1. The person who came up with the idea is actually very stupid.

2. The person speaking doesn't fully understand the idea, or all applications of it.

Guess which option is more commonly true?

Quite often it's just 3. They're obviously exaggerating, but still onto something.

Why is GOPATH a good idea? Can you explain it?

I guess the intention is to have all the packages you're using side-by-side (including the one you're working on)? That is not obviously a terrible idea, but as Go has more recently adding vendoring support, it clearly doesn't cover all the use cases.

If I'm using a third-party package, I almost always want to either install it in /usr/local (via Homebrew, say) for minimal hassle or vendor it into my project so the project is standalone. The GOPATH approach -- having external packages as peers of my project -- feels like a pretty distant third (although admittedly I'm not a Go user).

This feedback was given very early on in the development of Go, so the GOPATH behavior could easily have been changed, but clearly the language designers felt it was a good approach. I imagine it's a Plan 9 convention?

I don't understand the GOPATH criticism. For your C compiler to find included header files they need to be specified relative to /usr/include (or some other path contained in the list searched by your compiler).

For the go build tool to find Go packages they need to be given relative to (an element of) GOPATH (which is a colon-separated list of directories that is searched for Go package sources). To work on your own code inside your home directory while installing other packages using Homebrew, you'd set a GOPATH of $HOME:/usr/local. If you also want to consider packages shipped with the OS, you append a ":/usr" to that.

Lots of things work this way: @INC in Perl, sys.path in Python and many more. Why have people such issues understanding this concept when it's called GOPATH?

If I have a hello.c and hello.h in the same directory, and hello.c #includes "hello.h", I can "cc hello.c" and it always works just fine.

In Python, you have to add some empty __init__.py files to your project, but once you've done that, local imports work just fine, no environment variables needed.

GOPATH is unusual and not like other programming languages.

Once again, the two major use cases for imports are "stuff local to my project" and "stuff I've installed". GOPATH covers a third case, "other packages I want to use, that I haven't installed and haven't vendored". That doesn't seem important enough to warrant making "stuff local to my project" irritatingly fiddly.

If I download some Go program from Github, even if it has all its dependencies vendored, it doesn't work unless I put it in ~/go or set GOPATH. That's strange! Why wouldn't you want it just to work out of the box, wherever you put it?

I never said any of Golang's developers are stupid. On the contrary, they're clearly smart people. I like golang, and I think it's a great language.

But even smart people have bad ideas sometimes, and GOPATH is one of those.

I think I fully understand GOPATH.. I've been programming in GO every day for several years. But sure, go ahead and tell me what's so awesome about GOPATH, that can't be achieved by using the current working directory.

Nice article!

In my opinion, this level of abstraction may be going a little far:

    GOCMD=go
    GOBUILD=$(GOCMD) build
    GOCLEAN=$(GOCMD) clean
    GOTEST=$(GOCMD) test
    GOGET=$(GOCMD) get

It might seem like a good idea to constantize everything, but in practice, those commands are never going to change.

By just invoking them directly in the targets you'll come out with something more readable and no less maintainable:

    build: 
            go build -o $(BINARY_NAME) -v

That is a common Makefile design pattern. It allows you to call "make GOCMD=path/to/custom/prerelease/go" to run with an alternative go version, for instance. (But I agree, the other variables seem overkill.)

Don't we need to use "GOCMD ?= ..." in the makefile to allow overridding GOCMD from command line?

Yes, good catch.

The `?=' operator will only define the variable if it doesn't exist (and thus allows overriding via:

    GOCMD=/foo/bar/baz make

Depends on use case. There is no religion, make is just a tool for your needs. Your experience should dictate.

Mechanism, not policy — the good old and mostly hyped-out principle.

But you may want to add a compiler flag, or change the path of the binary. If the binary is called in multiple places it does make sense to constantize it.

Possibly, but by moving flags up to the constants you also run the risk of obfuscating things further. A little bit of repetition when it comes to compiler flags (especially in Go where there are relatively few) is a good trade for the added clarity it conveys.

Invoking a particular binary is valid, but even there, you only need one constant, not one for every subcommand.

Or stick in a wrapper.

I don't have much experience with makefiles but for shell scripts, I tend to think that your approach is correct as long as you're working with small or fairly trivial files.

If the file is small enough to see all at once (and your terminal window or editor isn't outrageously large), then mentally, you can take in the whole thing at once. Once the file starts to get even a little bit complicated, you'll want those abstractions.

Especially when dealing with Go, which embraces "a little bit of repetition"

In my recent cross-platform projects, I just make a build.go script that can handle multiple commands and then just have builders only use that. E.g.

    go run build.go build release

This crosses platforms, is easy to read and contribute to, is a much better language than make + CLI utils (or shell scripts), etc. It's not too much asking them to have "go" on the PATH. A similar approach could be taken w/ Python too. For most of this, there really isn't much code reuse needed, but it's often better to embed it than rely on external deps. Complex build script example: https://github.com/cretz/doogie/blob/8c53a266f35146a3c0d8f14....

camlistore does this too, but a Makefile calls "go run make.go" https://github.com/camlistore/camlistore/blob/master/Makefil...

You're missing half of what makefiles are for: only building what you need based on dependencies.

I never used Go so the following may not be technically correct, but in principle:

- The `build` rule should be `build: *.go` so that it builds only if a file has changed. - The `run` rule should have the `build` rule as dependency, so as to not rebuild if nothing has changed. Plus, it avoid to repeat the almost-identical build command (which is not very DRY)

Plus, I'm not convinced of the usefulness of having a $GOCMD variable (or $GOGET, $GOBUILD...). But again, never used go, no idea if the tooling is as tweakable as C (for which this kind of variable is used a lot in makefiles)

You really don't want to do that. I know what you mean but the go command handles the dependency stuff. Trying to shoe horn that into a Makefile is a disaster.

It's doable though: 'go list -f "{{.Stale}}" $PACKAGE' can be used to determine if a package needs to be rebuilt; this can be used in a make rule which touches a stamp file, and then the stamp file may be relied upon for other make rules.

My quibble with the example Makefile as written is that it doesn't utilise 'go install,' which I much prefer to 'go build.'

I've recently started using Makefiles for this reason, it cut my cache hit on checking if my docker image needs a rebuild based on dependencies from 1.8s to 0.03s. My workflow had me hitting this check probably 20 times/day, so it was a huge time save!

    IGNORE_FILES   := "Makefile\|.gitignore\|.dockerignore"
    DOCKER_CONTEXT := $(shell git ls-files -cmo --exclude-standard \
                                  --exclude-from=.dockerignore \
                            | grep -v $(IGNORE_FILES))

    .PHONY: build
    build: .build/docker_image

    .build/docker_image: .build $(DOCKER_CONTEXT)
        docker build -t $(IMAGE) .
        docker images -q $(IMAGE) > $@

    .build:
        @mkdir -p $@

I'm still figuring this stuff out, so I would love any feedback. :)

You can figure out what the dependencies of a file using go list, I used something like

  ${GOCMD} list -f "{{ .ImportPath }} {{join .Deps \" \" }}" ./cmd/$command | \
    xargs ${GOCMD} list -f "{{ if not .Standard }}build/mac/${command}: {{ range .GoFiles }}{{ \$.Dir }}/{{ . }} {{ end }}{{ end }}" ./cmd/${command} | \
    sed "s: /go/src/com.test/your/package/: :g"

...and then including the output of that in the Makefile. There are ways to make sure it gets rebuilt if anything changes, but you'll pay a performance price. It's rather similar to what happens with C, as far as I remember.

It was quite evil, somewhat redundant, and I eventually found make to complicate things for my application more than it was helpful, but sure, you can map modified go files to whatever tasks you might want to do.

EDIT: note that, unlike in C, go (usually) has a single static output file, so without the .o stage, the dependencies are quite dynamic - a change to a go file not included directly might be cause for a recompile. You can build intermediates, (I think either Buck or Bazel do that?) and link by hand, but that's very non-standard.

I don't think you can/need make to check timestamps for Go. `go build` will do it for you.

You do if you want to only run some tests sometimes or only trigger a new docker image if the binary actually changed.

go install caches build artifacts and only rebuilds when necessary (or when explicitly instructured to rebuild everything). go build builds from scratch every time—IIRC and if it hasn't changed since this I learned this. There's no need to attempt to reimplement go install's caching in make.

You can get build caching with "go build -i". It's a not very well known option.

There are a lot of problems with this approach and a few errors in the post, but I'm just going to highlight the most significant one:

> If the project uses CI/CD or just for consistency, it is good to keep the list of dependencies used in packages. This is done by the “deps” task, which should get all the necessary dependencies by “go get” command.

This is a very strong anti-pattern. Everyone should be using a dedicated version management tool (ideally dep[0], but there are others), which handles this seamlessly and behind the scenes. Introducing another place where dependencies are tracked and can be installed is a recipe for problems down the line.

[0] https://github.com/golang/dep/

Makefiles are great for *nix systems. But, Go is a first class language on Windows. For open source projects who want to have contributors cross platform... how can they use Makefiles effectively?

Install gnu make. I personally typically go with msys since it is slightly less intrusive and more windows friendly than cygwin. There is also the new Linux subsystem...

If you build under the new Linux subsystem, you'll get a Linux binary. That's not a good solution if you want to build a Windows binary.

Note I didn't say anything about what compiler will be used or commands actually run. Make just calls the shell. If the shell runs stuff that produces Windows binaries you get Windows binaries.

(Assuming they fixed the Linux subsystem in a way that execve(2) works on Windows binaries, which I heard in the first version they didn't have working, then there should be no problems.)

You can cross-compile, and I’ve done it a couple of times without any issues:

https://github.com/golang/go/wiki/WindowsCrossCompiling

Quick feedback:

There should be a BINARY_NAME target that 'run' depends on instead of repeating the build command.

The source files should be named as dependencies when they are inputs to a build step.

Anything that isn't a file should be listed as .PHONY target. Otherwise, make gets confused if you, for example, create a directory named 'test'.

If the goal of your makefile is to make your workflow simpler and easier to discover, consider adding a 'help' target that at least describes the interesting targets. It might also direct users to a more involved explanation if there are interesting things about your project they need to know about.

My only gripe with Makefiles is the lack of support on Windows. I work on several Go projects, all of which use a Makefile. It makes for a great experience on MacOS and Linux, but on Windows I have to break down the Makefile logic into powershell scripts or run raw `go test/build`. Both are fine, but just pointing out the fact that it's not a silver bullet if your project is intended to work across multiple platforms (like a CLI).

If you use Cmder[1] as your terminal, and you install the full package, make is bundled with it and works exactly as you'd expect it to. I do this with a couple of cross platform Go projects, and make works nicely on macOS, Linux, and Windows.

Of course, this is less useful if it's a team project, because then everyone will have to install Cmder (or at least msys) to use the Makefile on a Windows system.

[1]http://cmder.net/

Sure, I have `make` installed and I'm using cmder, but some things do not translate well in Windows.

https://cognitivewaves.wordpress.com/makefiles-windows/

MSYS2 offers a nice and relatively lean Unix-like build environment on top of Win32. It's especially convenient if your dependencies happen to be in their Pacman package repo (and if not, you can submit them to it...).

Does WSL not solve that?

If you do your build under WSL, you'll get a Linux binary. Which may or may not be OK, depending on what you're after.

Nope. Here: https://github.com/golang/go/wiki/WindowsCrossCompiling

Ah, that's a good point. You just need to remember to set the right environment variables. Unless you need to link against MSVC stuff, although I don't know if Go supports that anyway.

Doesn't look like go works with msvc yet - the following issue was a fascinating read (partly because of the optimism and persistence displayed, partly in a ohmigoditsatrainwreckcantlookaway-way ;)

https://github.com/golang/go/issues/20982

It isn't always as simple as setting some environment variables. None of the projects I work on will compile when told to compile for another OS_ARCH because of the dependencies they have.

This is gonna get me in trouble here, but I like using Python for build tool. The reason for it is because Python plays nicely on all 3 major platforms.

Same for me, and I have this mess: https://github.com/rakete/cute3d/blob/master/ninja_cute3d.py to show as proof. I am kind of cheating though, because I am not really building anything, I am generating a ninja build file and then use that to do the actual build.

Works flawlessly on Linux and Windows and is fast too!

Fair question. What do makefiles have that make them better than a good scripting language like python ?

Make will check if your dependencies need to be rebuilt, and will only rebuild them if they have changed / been updated.

This is hugely valuable because you can save build times by not needlessly re-compiling the whole world. Linking times are always going to remain the same regardless, but you can definitely make the process more efficient. An example from C/C++ is not generating new object (*.o) files for source files that haven't changed (and who's dependencies haven't changed).

Make also has a pretty standardized syntax and has a bunch of open source implementations. This may not seem to matter since Python has many implementations (as an example), but remember that make is just a program, not a full platform. Python may execute differently depending on the version of Python, the version of any library you're using in the process, or even on the architecture (okay, maybe not in Python so much, but in some scripting languages it could matter). Make only depends on the version of make that you install.

Lastly, make comes with parallelism in your build mostly for free. Calling `make -j4 all` will run up to 4 rules at a time in parallel. In some cases this won't work because your dependencies are tangled in a mess and parallelism can't be guaranteed, but it's a lot easier to get parallel builds for most projects with make compared to something like Python, which either has to use something like the multiprocessing library, or pray that the GIL doesn't bite. In my opinion, this is not something you should be baking yourself into your build system. Make has done it and it has been done right for years.

There are make competitors written in Python (and other good scripting languages). SCons is built in Python and its equivalent to makefiles are written in Python. Cons, from which SCons was forked/derived/something, is in Perl and Perl is its scripting language.

Plain old Python (or whatever) is missing any notion of dependencies (so you'll re-implement a topological sort in every large project), file identity (so, you're either checking timestamps or hashing them or something to know when files have changed), and probably some other stuff. You're kinda doomed to reimplement half of make poorly every time you start a new project, and you'll find out which half as your project grows.

Some of the things that "modern" build tools from a decade or two ago implement has since been implemented at the language level...so, while SCons can do things with remote dependencies, you wouldn't need/want that in a Go/Rust/JS/whatever project because they all already have their own dependency resolving package managers.

That's not to say make is ideal. But, a scripting language alone is probably less ideal.

References:

http://scons.org/

https://www.gnu.org/software/cons/

Python is a general-purpose language. Make is specifically focused on building artifacts. Python can do anything Make can do, but it's going to require more work on your part, which won't be as battle-tested as the stuff already in Make.

For example, run into a problem building an artifact with Make, and the internet is full of answers for you. Run into a problem with your own bespoke buildscript, and your troubleshooting gets harder.

I was always scared of make and it does have a learning curve, but then I spent the time to learn some basics and it's fine now - better than writing my own custom buildscripts.

Traditionally, intelligent dependency resolution with task selection based strictly on touched dependencies. When you add globbing and other expansion/interpolation features that's not trivial behavior.

But if dependency resolution is handled implicitly by Go's toolchain, then it becomes a closer race.

For tasks like the one the author describes, I prefer using just [1] over make. It's designed to be a command runner rather than a build tool, so you lose things like considering whether files have been modified when executing a target, but it includes some features that make it better than make for keeping a list of commonly-run commands.

In particular, it can: - list/show available commands - pass through command line arguments - run commands in a different directory - run with a different interpreter - has better error messages

[1] https://github.com/casey/just

> For tasks like the one the author describes, I prefer using just [1] over make.

Just really ought to be better known.

    # Go parameters
    GOCMD=go
    GOBUILD=$(GOCMD) build
    GOCLEAN=$(GOCMD) clean
    GOTEST=$(GOCMD) test
    GOGET=$(GOCMD) get
    BINARY_NAME=mybinary
    BINARY_UNIX=$(BINARY_NAME)_unix

A little bit of Clean Code education is needed here :) Why on earth do you want to introduce so unnecessary abstraction? If those would use a lot of switches which you don't want to duplicate and just possibly want it to change in one place, I would say OK. But you don't even use it that way:

    $(GOBUILD) -o $(BINARY_NAME) -v ./...

    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 $(GOBUILD) -o $(BINARY_UNIX) -v

So this doesn't make sense at all. You should write the commands as it is and it would be a lot easier to read an already hard-to-read syntax Makefiles anyway.

I'd say "GOCMD" would be sane if it were phrased "GO" instead. That's the convention for most commands in makefiles, stemming from the fact that if you need to invoke `make` recursively, you use `$(MAKE)` (provided automatically) to do it so you don't break nice things like -j.

Having a GOCMD may be useful, but not like this.

In bigger systems, you may alter the PATH in the CI env or part of the Makefile itself, or PATH differs from machine to machine, OS to OS, etc, or you want to say that GOCMD is either "gofork" or "go".

Regardless of the situation you'd want to use an absolute path to the executable so that if you run make in verbose mode you'd see "/usr/bin/go" or "/home/user/local/bin/go" or "/this/is/not/the/real/go" printed. Being super strict about this pattern will keep you sane.

I really liked makefiles and thought wouldn’t it be interesting if new features could be added to them so I made https://github.com/gabrielcsapo/build.sh it isn’t completely done as you can only run the entire pipeline but I really like the ergonomic that makefiles provided. (Written in node, also provided as a binary, node not necessary)

Aren't all the targets in OPs Makefile phony? Why not just have a simple shell script instead and not involve make at all?

I like Makefiles and Makefiles for Go are especially simple, but they don't play well with "go get", do they?

> I like Makefiles and Makefiles for Go are especially simple, but they don't play well with "go get", do they?

They don't (`go get` won't execute anything defined in a Makefile), and the approach that the author takes to vendoring is also going to cause problems.

It's possible to have a Makefile that serves as a shim for the standard and recommended Go workflow, but it's not very easy, and it's ultimately an uphill battle. It's also not going to be cross-platform, unlike the Go toolchain.

We don't use makefiles for cascading builds like you would in C and as such a lot of the power of Makefiles is lost in the golang world.

We do use them to specify "simple" repetitive tasks in a uniform way, with some minimal amount of order defined (for instance making sure go get is called before go test, or our code for embedding version numbers into binaries for builds).

[edit] An example from one of our public repos https://github.com/MediaMath/grim/blob/master/Makefile the codebase was one of our first times writing go so we aren't terribly proud of it, but the Makefile is representative of the kinds of things we do in make.

Just because you have a Makefile in your repository, it doesn't mean the package(s) aren't `go get`-able. As long as the package can be built with `go install`/`go build` it should be fine.

And there are cases where it is simply impossible for the package to be installed simply with "go get" or "go install". C'est la vie. The idea is not to unnecessarily introduce dependencies on things like make, not that you can remove the necessary dependencies on external tools.

Right, but then the value of the Makefile is pretty marginal.

The value of the Makefile is for automating repetitive dev tasks. Consumers of a golang CLI should use `go get` but this makes it easier for contributors who run tests, cut releases, ..

The authors example is clean/simple. Here's an example not so clean/simple example where we needed to do more complex stuff like build for all target platforms[1].

[1] https://github.com/cloudfoundry-community/stackdriver-tools/...

Been doing this since I started to use Go, with one twist: I set $GOPATH and have a vendoring target for each project, because Go _really_ needs per-project environments and saner dependency management... (dep is OK, but I need more control)

How do large Go projects outside of Google get built, anyway? Is it idiomatic to use Makefiles?

Inside Google, there are BUILD rules for Go, but it looks like they haven't been open-sourced in Bazel yet.

There are some released bazel rules for Go:

https://github.com/bazelbuild/rules_go

Nice! Are there any plans to document their existence in the Build Encyclopedia[0]? Or is the idea that each company should vendor the build rules for languages like Go?

[0] https://docs.bazel.build/versions/master/be/overview.html

Don't forget to use "go build -i", otherwise everything is compiled every time. With the -i flag you get caching, which can significantly speed up compilation.

I've seen many articles lately about using Make for NodeJS, Go, and other modern languages. I really do not understand it. Lots of these languages (Go and NodeJS) were designed to be cross-platform or at least easily compiled on different OS's.

Why use a platform specific tool and restrict yourself to Linux? I'd prefer people find a similar tool that is designed to work on any platform.

The “restrict yourself to Linux” bit seems very odd to me: make is one of the most portable build tools in existence. Even 20 years ago it was common to have makefiles which worked on a dozen Unix variants (not just *BSD & Linux), Windows, OS/2, VMS, BeOS, etc. and it’s gotten easier over time rather than harder.

Were you perhaps referring to platform-specific tools called by a Makefile?

Huh? I use make all the time on Windows. You don't even need Cygwin, MSYS, or Windows Subsystem for Linux. There are make binaries that work well, for example:

http://gnuwin32.sourceforge.net/packages/make.htm

What do u use for all the shell scripting and commands that usually go along with make

> Why use a platform specific tool and restrict yourself to Linux?

Make predates Linux by 15 years. It's not platform specific (indeed it is part of POSIX), and it is most certainly not restricted to linux.

I was using make on MS-DOS 2.1 way back in 1987. It's just not for Unix anymore.

That doesn't really matter, make is just a tool for your development environment, your code will be able to run on multiple architectures (which is what matters).

It works on OS X too. Linux plus OS X accounts for a lot of developers, though Windows probably still has the majority (but Windows is getting a Linux subsystem too!)

There are versions of gnu make that run on windows.

Is the word "be" missing in the title, or it's a joke?

It's a couple memes in one.

"I accidentally a word." is a meme

"He doesn't afraid of anything." is a meme

"Don't afraid of makefiles." combines the two and catches the eyes of those who know their memes.

The title a word.

The title what?

A word.

To answer you directly: It appears the author just forgot to include it.

Did you read the article? It's fairly clear that the author isn't a native speaker.

It looks like EGreg may also not be a native english speaker, from their own slight grammar error.

From the HN guidelines:

Please don't insinuate that someone hasn't read an article.

We may have been too hard on the GP, as I think he was just wondering whether it was accidental or the popular "doesn't afraid" meme.

Poe's law strikes again!

Cool! A variant of C language, makefiles... Now all I need is a tower PC with Windows NT 4.0 on it and I'll be all set!

On another thought... no. I'll stick in 2017 with modern tools and languages.

However you feel about go, it's not really a variant of C. The syntax is a bit similar, but there are a lot of fundamental differences.

This is a fair point. What I meant is that I think the usability niche for Go is similar to that of C. Is that a fair statement?

I was unfair to Go and this is because I seem to see many cases where Go is brought to where it does not fit.

One could argue that Go is a direct descendent of C, similar to the relation between Oberon and Pascal.

Reply on news.ycombinator.com