Hacker News Reader

Comment by rgbrenner | Hacker News Reader

There's another good use for Makefiles with go.

Gopath is one of the dumbest ideas I've seen. Multiple large projects under the same directory? Who thought that was a good idea. (and yes, I know you can create subdirectories, and mix all your packages together to create a giant mess.)

So I separate the projects into their own directories.. and use a Makefile to set gopath and run the go commands.

Still trying to figure out why golang tried to be innovative here.. What couldn't they use cwd like every other compiler I've ever seen?

View on HN

Yeah, that is what turned me off of Golang. It was kind of a "straw that broke the camel's back" thing, though. Go is just an EXTREMELY opinionated language, and if you don't like that then you won't like it. The Best Way to do everything is decided for you ahead of time, which irks me because it isn't always true. But I can see the frustration it grew from when I use a language like Ruby, which has a half-dozen nearly-identical ways to accomplish most simple tasks.

Anyways, you could muck around with symlinks and crap, but come on. If I can't simply mkdir/vim/compile/run a 'hello world' program, there's probably something wrong with the language.

> If I can't simply mkdir/vim/compile/run a 'hello world' program, there's probably something wrong with the language.

Like this?

`cat $HELLO_WORLD_PROG > /tmp/hello.go && go run /tmp/hello.go`

Yeah, like that!

But you know what I mean; multiple scattered workspaces, project folders on removable storage, etc. That sort of thing should be painless.

I've definitely had some other minor pain points with GOPATH, and I hate when veteran users of another language trivialize my pain points with their language, but I think all of the use cases you've described are all well within GOPATH's wheelhouse:

export GOPATH=$HOME/go:/media/USB-DISK:/foo/bar/baz

Now Go can find any projects in any of those workspaces. The only caveat is the projects have to be in a workspace, which is a directory with a src folder containing your project. I agree that this is a minor nuisance and I would design it differently, but Go's tooling is still far easier than pretty much anyone else's (it's actually what brought me to Go).

> but Go's tooling is still far easier than pretty much anyone else's (it's actually what brought me to Go).

Really?

Maybe that's true if you compare it to PIP (Python), Gems (Ruby) or NPM (JavaScript/NodeJS).

But I think that Cargo (Rust) got it almost perfectly right, and Opam+Ocamlbuild (OCaml) is very close.

Some other languages that make Go's tooling look awesome: C, C++, Java

Cargo is comparable to Go's tooling in my mind (in terms of ease of use; I think it's better in other categories), but figuring out how to use libraries (mod vs use) and organize a multi file project is a notorious pain point with no analog in Go. This means it's harder to get started with Rust than Go (language features notwithstanding), which is pretty much what we're discussing here.

I don't really know what to say about Ocaml. I don't have much experience, but most of it has been bad. It seems like you're supposed to use Makefiles, and an Opam file. The more I learned the less I liked, and it seemed to be the consensus that the build tooling was one of the language's weak points. I will say that installing packages from Opam was a painless experience. The build tooling wasn't the only thing that drove me away, but it was right up there with the poor standard library, ecosystem fragmentation, and low-quality documentation. Which is a shame because the language features look great.

With Go, all you need to know to get started is "go get".

>poor standard library

What's wrong with containers, batteries and core?

>go get

Wow, once I've tried go the package management was utterly broken, even cabal hell looked better than this. What, no package versions? Downloading directly from git repo, are you kidding me?

> What's wrong with containers, batteries and core?

Well, between those three and the actual std lib, there are four standard libraries, which defeats the purpose of having a standard library.

> Wow, once I've tried go the package management was utterly broken, even cabal hell looked better than this. What, no package versions? Downloading directly from git repo, are you kidding me?

`go get` isn't a package manager. Go's solution to dependency management is vendoring, which has its own suite of problems, but my comment was about getting newbies off the ground, and for that `go get` is suitable.

If by opinionated you mean it ignores everything that has been learned in the last 5 decades then yeah.

It's how Google does development. There's some big wins by having a mono repo for a large organization. It's a pita though for smaller projects

I don't think Google uses the standard Go conventions.

They use their build tool Blaze which I assume is very close to the open-source version, Bazel. Here's how Go works in Bazel: https://github.com/bazelbuild/rules_go

I seem to recall hearing that was finally being fixed, but a quick search of the docs suggests not. I think it's just GOROOT (another bizarre idea) rather than GOPATH that's no longer strictly required in the latest version of Go.

I agree with you, I don't understand why GOROOT couldn't just be the directory containing the compiler, and GOPATH couldn't just be the current directory. Maybe make me put some kind of workspace file there to identify it as the project root, that'd be fine.

> I seem to recall hearing that was finally being fixed, but a quick search of the docs suggests not. I think it's just GOROOT (another bizarre idea) rather than GOPATH that's no longer strictly required in the latest version of Go.

GOPATH is set to `~/go` by default, and if your projects use vendoring (which they should), you can almost completely ignore that it exists. Packages in `./vendor/` will take precedence over anything in your GOPATH.

Do most Go programmers put all their code under ~/go, or do they tend to set GOPATH on a project-by-project basis?

Most put it under ~/go. It's quick and easy.

Most set the GOPATH. It's quick and easy.

Quite a few current Go programmers will have something set other than `~/go` because that default only came in very recently, but you'll probably find that almost everyone uses `~/go` or `$GOPATH`.

One thing that you learn quickly when starting Go is that fighting the language's idioms (even where they're questionable) isn't going to be a good time.

Yeah, that's pretty much why I haven't learned Go. :)

Don't know about others, but since I run my projects in containers anyway, I'll usually use `direnv` on my host to set GOPATH to project directory, and then use the default GOPATH in go docker images (`/go`).

I could even do without direnv altogether, but I need it so that vim-go can report build errors on file saving.

I use virtualgo[0] so I can organise my work properly and not mix my own stuff up with loads of unrelated libraries

[0] https://github.com/getstream/vg

> if your projects use vendoring (which they should)

For versioning? Also:

1. I haven't seen projects on github using vendoring

2. It doesn't look like `go get` allows you to download a package directly into your vendor file

For my personal project [1] I use vendoring. But as of Go 1.8 it is still inconvenient as the main package has to be on GOPATH/GOROOT or the compiler will not find it.

To maintain it I use git-subrepo [2], a very convenient tool to embed the code into own tree.

[1] - https://github.com/ibukanov/ljdumpgo

[2] - https://github.com/ingydotnet/git-subrepo

Have a look at /coreos on GitHub -- flannel, dex and prometheus-operator all use vendoring, just looking at what's pinned. matchbox does also.

Thanks for the examples!

https://github.com/coreos/flannel/tree/master/vendor

For the Go tools to work consistently, your code has to be inside GOPATH, which means you cannot ignore it entirely.

You can often use relative package names (e.g. "go test ./..."), but it breaks whenever you use absolute ones, and the Go toolchain is based on this assumption being true. My experience is that editor plugins (e.g. the Go plugin for VSCode) don't like this.

I use hardlinks (the "hln" third-party utility on macOS) to get around this, which means I can have all my projects under a sane directory structure under ~/Projects.

After struggling a lot, I think, I finally understood best use for GOPATH with Makefile based projects. I use it as follows:

I have ~/go for all 3rd party code that I did not develop. My personal projects live in a different directory with a different structure (including, configure.ac, Makefile.am, etc.). My projects tend to use multiple languages, so I need Makefile support.

I fix GOPATH in project specific Makefiles with ~/go as the first directory followed by project specific directory for Go code. For example, I add the following in Makefile.am of every project.

GOPATH := $(HOME)/go:$(CURDIR)/go

One cool result of this is: When I do go-get of any 3rd party package (either through command-line or as part of a rule from Makefiles), Go downloads them into ~/go directory (it picks the first directory in GOPATH or defaults to ~/go). This leaves my project specific directories free of 3rd party code -- which is what I want.

End result is, ~/go ends up with 3rd party code from across all projects and it doesn't include my project code, so can be deleted any time and re-created automatically by Makefile rules.

How do you deal with conflicting versions?

Honestly, I didn't face that issue till now.

It is also my biggest gripe with Go and it's not quite easy to workaround. I am using a Makefile approach for that: https://vincent.bernat.im/en/blog/2017-makefile-build-golang

Perhaps Dave Cheney's gb would interest you; it does not use GOPATH. https://getgb.io/

GOPATH exists to tell the go tool where to find the libraries that go get installs. You can have multiple GOPATH entries in there if you want to segregate things. It's basically PYTHONPATH with a few extra bits.

One of the reasons is that many of the developers of Go subscribe to a monorepo philosophy. GOPATH enables and encourages that practice.

You can work around the GOPATH issue by using Docker. If you build go within a Docker container, you can structure the folder however you like and it won't affect the host.

The only downside is that your text editor might stop offering some features when editing a go file, since the import paths might not point to the actual location.

The only downside being well having to use docker to work in a modern language

I don't think you have to go that far. I think something like https://direnv.net/ should work too. Although, I do agree that having this restriction in the first place is bizarre.

> Multiple large projects under the same directory

Sorry to be snarky, but everything, eventually, is under one directory. "/".

There seems to be two kinds of folks working with Go: those who want to do it their way, and those who want to do it the Go way. I'm in the latter group.

> ... I separate the projects into their own directories

So do I, but I just don't get to pick that directory. Everything is in $GOPATH. $GOPATH/src/github.com/:org/:package is my directory for any given project. That project has a /vendor directory with its dependencies. Everything is contained within my project's directory. It is in source control. Everything just works with the Go tooling.

> ... use a Makefile to set gopath ...

If you don't fight the system, you don't need to set GOPATH per project. I totally get there is something that maybe I just don't understand or am totally missing, but in _years_ of working with Go on many dozens of applications, I've never had to alter my GOPATH (aside from the one application that a team at our company did where they put their source control at the $GOPATH/src directory, which was a pain in the butt and is not the way to do things).

> Still trying to figure out why golang tried to be innovative here...

Yeah, it was weird for me at first. I accepted it as the way of things and my life got better.

And to harken more back to the heart of the thread: makefiles. While I just don't understand the need for separate GOPATHs, I can see some value for complicated build or test commands. For the most part, `go test -race` and `go build` is what I need 90% of the time. Mostly, it is the build system that needs the equivalent of `make test` and `make build` to ensure flags are set and such.

I believe the idea behind it is that you should only deploy one app per container. Perhaps developing locally with Kubernetes would be a simpler path.

Whenever someone says "X is one of the dumbest ideas" there are two possibilities:

1. The person who came up with the idea is actually very stupid.

2. The person speaking doesn't fully understand the idea, or all applications of it.

Guess which option is more commonly true?

Quite often it's just 3. They're obviously exaggerating, but still onto something.

Why is GOPATH a good idea? Can you explain it?

I guess the intention is to have all the packages you're using side-by-side (including the one you're working on)? That is not obviously a terrible idea, but as Go has more recently adding vendoring support, it clearly doesn't cover all the use cases.

If I'm using a third-party package, I almost always want to either install it in /usr/local (via Homebrew, say) for minimal hassle or vendor it into my project so the project is standalone. The GOPATH approach -- having external packages as peers of my project -- feels like a pretty distant third (although admittedly I'm not a Go user).

This feedback was given very early on in the development of Go, so the GOPATH behavior could easily have been changed, but clearly the language designers felt it was a good approach. I imagine it's a Plan 9 convention?

I don't understand the GOPATH criticism. For your C compiler to find included header files they need to be specified relative to /usr/include (or some other path contained in the list searched by your compiler).

For the go build tool to find Go packages they need to be given relative to (an element of) GOPATH (which is a colon-separated list of directories that is searched for Go package sources). To work on your own code inside your home directory while installing other packages using Homebrew, you'd set a GOPATH of $HOME:/usr/local. If you also want to consider packages shipped with the OS, you append a ":/usr" to that.

Lots of things work this way: @INC in Perl, sys.path in Python and many more. Why have people such issues understanding this concept when it's called GOPATH?

If I have a hello.c and hello.h in the same directory, and hello.c #includes "hello.h", I can "cc hello.c" and it always works just fine.

In Python, you have to add some empty __init__.py files to your project, but once you've done that, local imports work just fine, no environment variables needed.

GOPATH is unusual and not like other programming languages.

Once again, the two major use cases for imports are "stuff local to my project" and "stuff I've installed". GOPATH covers a third case, "other packages I want to use, that I haven't installed and haven't vendored". That doesn't seem important enough to warrant making "stuff local to my project" irritatingly fiddly.

If I download some Go program from Github, even if it has all its dependencies vendored, it doesn't work unless I put it in ~/go or set GOPATH. That's strange! Why wouldn't you want it just to work out of the box, wherever you put it?

I never said any of Golang's developers are stupid. On the contrary, they're clearly smart people. I like golang, and I think it's a great language.

But even smart people have bad ideas sometimes, and GOPATH is one of those.

I think I fully understand GOPATH.. I've been programming in GO every day for several years. But sure, go ahead and tell me what's so awesome about GOPATH, that can't be achieved by using the current working directory.

Reply on news.ycombinator.com