Golang Dependency Management Best Practice
In Golang, we can specify open source libraries on GitHub as dependencies. For example:
import "github.com/RichardKnop/somelibrary"
This will try to look for a branch based on your Go version and default to master if I understand correctly.
So there is no way to import a specific release of a dependency, e.g.:
import "github.com/RichardKnop/somelibrary#v1.4.8"
What is the best practise to manage dependencies in Go then?
I can see two approaches.
I. Version Modules
Is it to create new modules for major versions with breaking changes?
For example, my Go library could define modules v1 and v2 so then you could do:
import "github.com/RichardKnop/somelibrary/v1"
Or:
import "github.com/RichardKnop/somelibrary/v2"
Based on what you need. Any changes made to v1 or v2 would be required not to break any APIs or working functionality.
II. Forking
This would give you a complete control over a version of external dependency your Go code requires.
For example, you could fork github.com/RichardKnop/somelibrary into your own GitHub account and then in your code do:
import "github.com/ForkingUser/somelibrary"
Then you would have to fork all external dependencies which seems a bit overkill. However it would give you total control over versions. You could keep your forks at a version you know is working with your code and only update forks once you have checked that new releases of dependencies do not break anything.
Thoughts?
Feb. 2018: the vendoring approach presented below (in 2015/2016) might end up disappearing if vgo is integrated to the toolchain.
See my answer below.
August 2015 edition: Go 1.5 comes with a built-in (but still experimental) vendoring support. Setting the environment variable GO15VENDOREXPERIMENT will make go build and friends look for packages in ./vendor directory as well as GOPATH. See VonC's answer and the design document for more details.
III. Vendoring
AFAIK, this is the most widely used way of ensuring that your builds are reproducible and predictable. The Go team itself uses vendoring in their repo. The Go team is now discussing the unified dependency manifest file format. From the Go toolchain developers mailing list:
In Google’s internal source tree, we vendor (copy) all our dependencies into our source tree and have at most one copy of any given external library. We have the equivalent of only one GOPATH and rewrite our imports to refer to our vendored copy. For example, Go code inside Google wanting to use “golang.org/x/crypto/openpgp” would instead import it as something like “google/third_party/golang.org/x/crypto/openpgp”.
(...)
Our proposal is that the Go project,
officially recommends vendoring into an “internal” directory with import rewriting (not GOPATH modifications) as the canonical way to pin dependencies.
defines a common config file format for dependencies & vendoring
makes no code changes to cmd/go in Go 1.5. External tools such as “godep” or “nut” will implement 1) and 2). We can reevaluate including such a tool in Go 1.6+.
Note: June 2015, the first support for vendoring appears in Go 1.5!
See c/10923/:
When
GO15VENDOREXPERIMENT=1is in the environment, this CL changes the resolution of import paths according to the Go 1.5 vendor proposal:
- If there is a source directory
d/vendor, then, when compiling a source file within the subtree rooted atd,import "p"is interpreted asimport "d/vendor/p"if that exists.- When there are multiple possible resolutions, the most specific (longest) path wins.
- The short form must always be used: no import path can contain “
/vendor/” explicitly.- Import comments are ignored in vendored packages.
Update January 2016: Go 1.6 will make vendoring the default.
And as detailed in the article "MOST GO TOOLS NOW WORK WITH GO15VENDOREXPERIMENT":
1.6 brings support for
/vendor/to most tools (like the oracle) out of the box; use the Beta to rebuild them.
- issue 12278 has been resolved.
-
there still is an issue with.goimports, and there is a CL that can be cherry-picked
Update August 2018: this (vgo presented below) is now implemented with Go 1.11 and modules.
Update Feb. 2018, 3 years later.
There is a new approach developed by Russ Cox, from the core Golang development team.
The vgo project.
go get -u golang.org/x/vgo
This proposal:
- keeps the best parts of
go get,- adds reproducible builds,
- adopts semantic versioning,
- eliminates vendoring,
- deprecates
GOPATHin favor of a project-based workflow, and- provides a smooth migration path from
depand its predecessors.
It is based on a new MVS ("Minimal Version Selection") algorithm:
You can see:
- the first articles in Go & Versioning
- the discussion in this thread
- the analysis of that thread by Jon Calhoun
- the counter-point by Sam Boyer in "Thoughts on vgo and dep":
He was leading the private initiative ofgo dep - the first debate (YouTube) between Russ, Sam and Jesse Frazelle!
May 2018: Artifactory proposes a vgo proxy
The recent release of Artifactory 5.11 added support for vgo-compatible Go registries (and go proxy) giving the community a variety of capabilities when developing with Go.
Here are just a few of them:
- Local repositories in Artifactory let you set up secure, private Go registries with fine-grained access control to packages according to projects or development teams.
- A remote repository in Artifactory is a caching proxy for remote Go resources such as a GitHub project. Accessing a go proxy through Artifactory removes your dependency on the network, or on GitHub since all dependencies needed for your Go builds are cached in Artifactory and are therefore locally available. This also removes the risk of someone mutating or removing a dependency from version control, or worse, force-pushing changes to a remote Git tag thus changing what should be an immutable version which can create a lot of confusion and instability for dependent projects.
- A virtual repository aggregates both local and remote Go registries giving you access to all Go resources needed for your builds from a single URL, hiding the complexity of using a combination of both local and remote resources.