Dependencies and vendoring in Go | Hacker News Reader

Dependencies and vendoring in Go

groups.google.com

113 pointsrgarcia29 comments

I think that the idea itself sounds grand. No doubt there'll be some implementation hurdles, but it all makes sense.

But:

> godep already uses JSON, as do Go tools themselves (e.g. “go list -json fmt”), so we’ll probably want something like godep’s JSON format…

Seriously, the only things good about JSON (and it is good) are that it's so widely supported and that it has a clean representation of maps/dicts/hashes/whatever-you-call-them. Take a look at the two lines of text config in the article, vice the twelve lines of JSON.

What's wrong with nice, clean formats line text lines, s-expressions, even .ini? JSON is better than XML, but that's damning with faint praise.

I'd say JSON is more on the level of s-expression than an ini file. JSON objects are extremely regular (your thing's either an object or a literal), more-so than your standard 2-level ini file. Plus there's a lot of stuff.

Sure, JSON might be a bit more verbose for the "Hello World" scenario, but in mid-range sizes, it's still pretty clear (compared to the verbosity of XML) and doesn't require custom tooling to deal with (like ini files).

There are some formats with less braces (replaced with indents and with less key stuff) that are (I think) isomorphic to JSON that would be nicer to use for everyone involved, though.

agreed, I think INI-style (specifically https://code.google.com/p/gcfg) is a better choice for simple config files. That said, they might be anticipating more complex configs in the future.

TOML seems to be a popular INI successor, however the idea around suggesting JSON (or native Go) is that it can be easily parsed with standard libraries.

Obvious solution is to add toml to the std lib. :) only half joking. Toml rules.

I wouldn't be surprised if that happened, it seems like a good fit. Is there a standard or RFC?

s-expressions are unreadable for most people. .ini can't represent complex structures or even simple ones like lists or maps. Things like YAML exist but aren't better than JSON by enough to justify a proliferation of formats. JSON everywhere would be a win.

This is more or less a tool-supported version of what we do at work--I've posted about it a few times, to sneering response.

The config file thing would hopefully be nice and small/simple so it's not a hassle to use. I think their '“rsc.io/x86/x86asm” with revision af2970a7819d' format would be more pleasant than JSON, but if they need more detail than just a repo path and a revision, JSON may be the way to go.

Don't get too excited. This proposal is more a codification of the "easy" part of managing dependencies for top level/binary packages that much of the community already does. This proposal does not (and isn't intended) to cover how dependencies work when vending libraries and does not address shared package and diamond shaped dependencies.

That's not to say having a standard for vendoring isn't a good thing.

Things seem extremely nice in go when you start out in go and see "go get", but then you'll quickly ram into not wanting the latest master, and then get extremely puzzled about how to do library dependencies.

I wish I saw some sort of consensus on library dependencies in that thread. Too much of our work build relies on setup scripts.

I thought this was a decent proposal:

https://github.com/mattes/go/blob/fix-dependencies/fix-depen...

Discussed here:

https://www.reddit.com/r/golang/comments/2xeuu0/allow_revisi...

It's decent, but the complication introduced from the possibility of different tags in different source files poses a problem. I'd rather have all my deps in one place.

Additionally, it completely breaks if you depend on a package which depends on a specific version of another package. You are stuck using that version or forking your dependency to update the dep.

Seems to me, it was perhaps the worst decision Go made, not to build version numbers into the import syntax and file path layout from day one.

The issue of disappearing vendors can largely be solved using godep and distributing binaries, but I don't feel like any language I've used has an effective mechanism for coupling unit testing with dependency management.

Wouldn't it make sense to be able to test your code in advance of updating a dependency to see the how's and why's of its failures?

> The issue of disappearing vendors can largely be solved using godep

Godep isn't "canon". There lies the problem. There should be no need for godep,the library dependecy issue should be solved once and for all by the go team so everybody uses the same library dependency manager. Vendoring dependencies solves nothing. Should I bundle these dependencies If I put my libs on github? and what if a dep has a bug? how do I know what version was used if I want to fix the lib myself? All that stuff stinks. I like Go but these are serious issues that will backfire, because other plateforms have been there and done that...

Have other platforms done this? I'm having trouble thinking of a language toolkit that has dependency management built by the core team. All of these solutions are third-party, they have just become the de-facto standard:

Java uses Maven or Gradle: third party

Python uses pip: third party

Ruby uses gems: third party

Node.js uses npm: third party

I'm sure I'm missing some, but this seems to be the norm. Go just hasn't been around long enough for a single third-party solution to become the de facto standard.

I think this is something of a shibolleth for a certain type of programmer:

* Thinks that dependency management of third party libraries is an important problem in their project.

* Demands that there be one centralized repo and method for that. No alternatives!

Maybe it shows that I am a C fan, but I kind of skew on the opposite of this spectrum. Really the most important part of your project is how you import a library? Really? Even out of chaos, like a language with no native support for it, it's a small chore and you get over it quickly. But I am not one of the types that agonizes over it, as I guess go people are.

I'm a C fan, too, but not when it comes to dependencies. In a world where using someone's library typically requires one or more of (a) having everyone compiling your code also manually compile the library, (b) hoping the library has found its way into N different OS package managers, and (c) vendoring requiring manual maintenance, you end up with a lot of real world C and C++ code reimplementing basic things that would be better done in a library, because importing a library is too much work. This sucks - certainly it's possible for libraries to be too small, dependency graphs too convoluted, and you can see some of that on npm, but C/C++ are so far on the other end of the spectrum that it's hardly worth thinking of.

Not trying to say it isn't painful, just that people live with it and are still productive. And by not mandating a single one you get several experiments. Autoconf could be one. pkg-config another. The *bsds do interesting things with a particular dialect of makefiles. Even Microsoft has a few: the wdk build system, vs/msbuild, nmake... All of these have ups and downs but few of them are outright "wrong", we live with them and I wouldn't want to see any one mandated...

What's wrong with reimplementing basic things? Dependencies are points of fragility; I'm happy to pay a little extra redundancy in exchange for greater resilience.

> I'm having trouble thinking of a language toolkit that has dependency management built by the core team.

Rust + Cargo.

Ceylon + Herd

> Python uses pip: third party

FWIW pip is now bundled with Python unless specifically unbundled (e.g. by linux distros)

Scala + SBT. Best thing ever.

Checking in your deps handles that. If your tests don't pass, don't check in the change.

But this requires source control systems that can store the files for many projects at a time. (See today's other discussion about git.) [1]

[1] http://bitquabit.com/post/unorthodocs-abandon-your-dvcs-and-...

Depending on specific versions of dependencies should also handle that, without needing to check in all dependencies. This is the approach generally taken when using Maven and related tools.

The only difference is that finding the dependencies can be a pain if they changed location, or github is down, or the specific version isn't a specific git commit or something like that.

Since there's no real difference, it doesn't really matter if you check in deps or have specific versions in a config file, either solution solves the underlying problem of having non-precise dependencies (i.e. just a github repo, or a version selector like >= 2.6)

I've been doing this with a simple alias and a src directory for quite some time:

alias go='GOPATH=`pwd`:$GOPATH go'

glide works really well for this: http://technosophos.com/2014/09/22/how-glide-solves-go-vendo...

Reply on news.ycombinator.com