If you have questions or input those who are working on this problem are listening.
Why is the protocol side of package management not a solved problem already?
At least Go reuses the git protocol to its advantage. But the general problem of package management applies to a lot of things.
"There is a package with a name and a description. That package has versions, which can be greater/lesser than others. Those versions may have deltas from one-another. They contain one or more files. Those files must have their integrity checked. etc, etc..." -> Why does every single language reimplement that?
Nevermind languages actually. Why does everyone reimplement that? Firefox/Chrome extensions, Android/ios app stores, a bazillion different game addons and app-specific stores...
We have a mostly-identical problem being solved a million different ways. pypi, npmjs, glide, they should be running the same server software. It'd be a first step towards standardization and avoid a bunch of mistakes repeated every time a new solution comes out.
Is anyone even trying to do something about this? (and before anyone says "do it yourself", a package management protocol is something I used to work on, not anymore. It keeps getting worse - it's not just a technical problem, also a political one.)
https://medium.com/@sdboyer/so-you-want-to-write-a-package-m...
Discussion https://news.ycombinator.com/item?id=11088125
Sam is currently working on a library for Go https://github.com/sdboyer/gps
First, different languages have subtleties that can make a different in package management. For example, the way nested dependencies can work in JavaScript, Go, and PHP are all different. Or, the way you can scan a codebase to determine things is difference.
It would be great if different languages had virtually the same interface. That way metadata and interaction could be about the same. Like diving a car from different manufacturers. In many ways the different package managers are similar enough for programming languages were sorta there (but not all that well).
Second, there is the cultural problem. Many languages are designed on islands from others. Reinventing things, trying different things, and wanting to be their own special flower. Sometimes this is good because a new language is not like the others and that can be a good thing. Other times you reinvent a fairly common wheel.
I'm not aware of anyone solving this in a common way.
Several efficient solvers have been implemented that supports this input format. You can read about it more in http://opam.ocaml.org/doc/Specifying_Solver_Preferences.html...
[1]: https://www.youtube.com/watch?v=E-gtFnbHcv0&list=UUP9g4dLR7x...
[2]: http://ocaml.org/meetings/ocaml/2014/ocaml2014_17.pdf
This is at least used in OPAM (the OCaml package manager) and is also available for apt. If you are implementing a SAT solver for your package manager nowadays, you are doing it wrong. :)
This is a statement that doesn't always apply.
For example, some packages don't have versions, they only have a single version: the latest version. It's a moving target, but it's there. Older versions are just an unimportant and unsupported implementation side-effect.
Of course, many packages do have versions, but this one factor makes a big difference on how you look at vendoring.
Second, just because some people do this doesn't mean it's a good idea. If you're writing a full application and just want to distribute it, do whatever you want with your versioning (although using a dependency management tool to distribute your webapp or CLI tool is probably not a great idea in the first place), but if you are writing something to be included in another project please use a sane versioning system. Anything else is selfish and disrespectful of your users.
If you want to change version (major), you need to create a new package.
The advantage is that you do not have to download a manifest such as package.json or whatever. It also provides a constraint on library authors to provide a backward compatible API for their published package.
Eventually, I guess we could use go/types to enforce/check the API backward compatibility requirement.
The main issues for now are:
* make sure that packages importing a same vendored one are able to use the same latest minor version in use.
* allow a project based approach to go get for people who need reproducible builds (or alternatively allow for multiple $GOPATH and make being able to switch between them easy)
I think of OpenStack here. Multiple times per year they produce releases because many people need them. Yet, the CI/CD system keeps it in such shape you can always run the tip of master and some people do that.
Both options need to be available.
To not have release versions means you're not willing to support a certain class of users. It says something about your intent.
I think this is a mistake. Packaging should come from a central server (that can be overriden with a mirror). Allowing downloads from adhoc git repositories is a real pain. First, it makes mirroring the package repository impossible. Second, it makes it difficult for companies to disallow certain packages in a centralized fashion. Third, it makes distro packaging a royal pain. And finally Git simply wasn't built for the this kind of work, compressed tarballs happen to work great for this purpose. Git's mutable history also presents other challenges for pinning to specific versions of packages. Simply put, sometimes a central authority has advantages.
That all said, vendoring alleviates most of these problems for apps and some of them for libraries. Reusing tools does make it a little easier to get started with too. I'd hate to see these features removed in whatever go get becomes in the long term.
Personally I don't think we need yet another package manager with painful external dependencies, non-reproducible builds, conflicts, lack of rollbacks, etc. We have that with "go get". What we need is to actually solve all those thing. Make sure there are no fragile external dependencies, but everything is in the package manager, even gcc, binutils. You get the idea.
There are a host of other reasons that we could happily debate for years (no thanks), but bottom line: you can't use it everywhere you can write/compile Go code.
(I've tried Glide but it slowness and few bugs made me move to Govendor. Nothing personal, it just didn't work out that well for me.)
A few questions:
- Is the intention desincentive the use vendoring?
- Is the intention to make a single solution, and then deprecate existing tools (Glide, Govendor and like)?
- Do you have the intention to make a central registry for Go packages? Or the solution will use GitHub and like, as it does today? It would be not backward compatible, but the document don't make this clear.
I understand how operating system package managers deal with things differently than application developers. I've heard of problems with the way PHP apps are handling their dependencies these days and how Linux distro package managers aren't happy about it.
I'm curious to know more in this case.
But there have been several regarding the challenges Fedora has had as a large distribution while trying to package many Go apps, each with potentially hard requirements on different versions of the same golang package to import. Most of the conversation about it is in the golang sig (special interest group) that I've linked to their mail.
May I suggest you subscribe, send an email out introducing yourself, and ask what issues we are facing? I suspect it would be extremely well received having someone working on a language specific solution asking on the Fedora go sig mailinglist. Just a thought.