Organize local code in packages using Go modules

I can not find a way to factor out some code from main.go into a local package when using Go modules (go version >= 1.11) outside of $GOPATH.

I am not importing any external dependencies that need to be included into go.mod, I am just trying to organize locally the source code of this Go module.

The file main.go:

package main

// this import does not work
import "./stuff"

func main() {
    stuff.PrintBaz()
}

The file ./stuff/bar.go (pretending to be a local package):

package stuff

import "log"

type Bar struct {
    Baz int
}

func PrintBaz() {
    baz := Bar{42}
    log.Printf("Bar struct: %v", baz)
}

The file go.mod (command go mod init foo):

module foo

go 1.12

When executing go run main.go:

If I import "./stuff", then I see build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.
If I import "stuff", then I see build command-line-arguments: cannot load stuff: cannot find module providing package stuff.
If I import stuff "./stuff" with a package alias, then I see again: build command-line-arguments: cannot find module for path _/home/<PATH_TO>/fooprj/stuff.

I can not find a way to make local packages work with go modules.

What's wrong with the code above?
How can I import local packages into other Go code inside a project defined with Go modules (file go.mod)?

Solution 1:

Module structure

The most common and easiest approach is:

Use a single go.mod per repository, and
Place the single go.mod file in the repository root, and
Use the repository name as the module path declared in the module line in the go.mod
- (If you are using a custom import path such as me.io/mymod rather than using a VCS host based import path, then you would use the custom import path instead of the repository name in your go.mod).

For example, if your repo is github.com/my/repo, then you would place a single go.mod in the repo root, with the first line reading module github.com/my/repo. That can be created by cd'ing to the repo root and running go mod init github.com/my/repo.

Following this helps you stay on the happy path with modules, and it avoids multiple subtleties.

Russ Cox commented in #26664:

For all but power users, you probably want to adopt the usual convention that one repo = one module. It's important for long-term evolution of code storage options that a repo can contain multiple modules, but it's almost certainly not something you want to do by default.

There is much more about multi-module repositories in the "Multi-module Repositories" FAQ section on the modules wiki. Those 6 or so FAQs in that section should be read in their entirety by anyone considering veering off the recommendation above.

Arranging packages within a module

Once you have set up your go.mod, you can arrange your packages in directories however you see fit in directories underneath the directory containing the go.mod, as well as in the directory with the go.mod. Three good articles about how to arrange your code in packages:

https://rakyll.org/style-packages/
https://medium.com/@benbjohnson/standard-package-layout-7cdbc8391fc1#.ds38va3pp
https://www.goinggo.net/2017/02/design-philosophy-on-packaging.html

Those articles are classics that pre-date the introduction of modules, but the philosophies in them still apply to how to arrange your packages within a module.

Importing other packages in the same module

When importing another package with modules, you always use the full path including the module path. This is true even when importing another package in the same module. For example, if a module declared its identity in its go.mod as module github.com/my/repo, and you had this organization:

repo/
├── go.mod      <<<<< Note go.mod is located in repo root
├── pkg1
│   └── pkg1.go
└── pkg2
    └── pkg2.go

Then pkg1 would import its peer package as import "github.com/my/repo/pkg2". Note that you cannot use relative import paths like import "../pkg2" or import "./subpkg". (This is part of what OP hit above with import "./stuff").

Modules vs. repositories vs. packages vs. import paths

A Go module is a collection of related Go packages that are versioned together as a single unit. Modules record precise dependency requirements and create reproducible builds.

Summarizing the relationship between repositories, modules, and packages:

A repository contains one or more Go modules (most often exactly one module in the repository root).
Each module contains one or more Go packages.
Each package consists of one or more Go source files that all reside in a single directory.
Go source code:
- declares its own package with a package foo statement.
- automatically has access to other Go source code in the same package.
- imports code from another package via an import path supplied in an import statement such as import "github.com/my/repo/pkg1". The import path always starts with the module path of that package, regardless of whether that package is in the same module or a different module.

Solution 2:

First you have to choose a name for your project and write it to go.mod file. This name belongs to root directory of the project. Each new package you create must be located inside its own subdirectory and its name should match directory name.

go.mod:

module myprojectname

or (preferred way, see @typical182's answer below for details)

module github.com/myname/myproject

Then import your project's packages like:

import myprojectname/stuff

import github.com/myname/myproject/stuff

Files of package stuff should be located inside project's stuff directory. You name these files as you like.

Also it's possible to create deeper project structure. For instance, you decided to separate source code files from other ones (like app configs, docker files, static files, etc...). Let's move stuff directory inside pkg, every go file inside pkg/stuff still have stuff package name. To import stuff package just write:

import myprojectname/pkg/stuff

Nothing stops you from creating more levels in the hierarchy like github.com/myuser/myproject/pkg/db/provider/postgresql, where:

github.com/myuser/myproject - project name.
postgresql - package name.
pkg/db/provider/postgresql - path to the package relative to project's root.

You can read more about go modules here: https://github.com/golang/go/wiki/Modules

Check out this repository to get useful information about various patterns used in project organizing: https://github.com/golang-standards/project-layout If you go inside pkg directory you will find out which open source projects use pkg directory in their structure.