How to embed files into Go binaries

I have some text file that I read from my Go program. I'd like to ship a single executable, without supplying that text file additionally. How do I embed it into compilation on Windows and Linux?

Solution 1:

===== Edit Jan 2021 =====

Starting with Go 1.16, released in Feb 2021, you can use the go:embed directive:

import "embed"

//go:embed hello.txt
var s string
print(s)

//go:embed hello.txt
var b []byte
print(string(b))

//go:embed hello.txt
var f embed.FS
data, _ := f.ReadFile("hello.txt")
print(string(data))

===== Original answer ======

Since Go 1.4, you can use go generate if you need more flexibility.

If you have more than one text file or the text file may change you might not want to hardcode the text file but include it at compile time.

If you have the following files:

main.go
scripts/includetxt.go
a.txt
b.txt

And want to have access to the contents of all .txt files in main.go, you can include a special comment containing a go generate command.

main.go

package main

import "fmt"

//go:generate go run scripts/includetxt.go

func main() {
    fmt.Println(a)
    fmt.Println(b)
}

The go generate command will run the script after go:generate. In this case it runs a go script which reads all text files and outputs them as string literals into a new file. I skipped the error handling for shorter code.

script/includetxt.go

package main

import (
    "io"
    "io/ioutil"
    "os"
    "strings"
)

// Reads all .txt files in the current folder
// and encodes them as strings literals in textfiles.go
func main() {
    fs, _ := ioutil.ReadDir(".")
    out, _ := os.Create("textfiles.go")
    out.Write([]byte("package main \n\nconst (\n"))
    for _, f := range fs {
        if strings.HasSuffix(f.Name(), ".txt") {
            out.Write([]byte(strings.TrimSuffix(f.Name(), ".txt") + " = `"))
            f, _ := os.Open(f.Name())
            io.Copy(out, f)
            out.Write([]byte("`\n"))
        }
    }
    out.Write([]byte(")\n"))
}

To compile all .txt files into your exectutable:

$ go generate
$ go build -o main

Now your directory structure will look like:

main.go
main
scripts/includetxt.go
textfiles.go
a.txt
b.txt

Where textfiles.go was generated by go generate and script/includetxt.go

textfiles.go

package main 

const (
a = `hello`
b = `world`
)

And running main gives

$ ./main
hello
world

This will work fine as long as you're encoding UTF8 encoded files. If you want to encode other files you have the full power of the go language (or any other tool) to do so. I used this technique to hex encode png:s into a single executable. That requires a minor change to includetxt.go.

Solution 2:

Use go-bindata. From the README:

This tool converts any file into managable Go source code. Useful for embedding binary data into a go program. The file data is optionally gzip compressed before being converted to a raw byte slice.

Solution 3:

Was looking for the same thing and came across esc: Embedding Static Assets in Go (by 19 Nov 2014) where author, Matt Jibson, is evaluating 3 other popular packages that claims to do file embedding:

1. rakyll/statik
2. jteeuwen/go-bindata (and the new official go-bindata/go-bindata and another improved one kevinburke/go-bindata)
3. GeertJohan/go.rice

and explain why he eventually come up with his own package:

4. mjibson/esc

So after briefly trying them all (in that order) I've naturally settled on Matt's esc as it was the only one that was working out of the box with necessary for me functionality (HTTPS service in a single executable), namely:

1. Can take some directories and recursively embed all files in them in a way that was compatible with http.FileSystem
2. Can optionally be disabled for use with the local file system for local development without changing the client's code
3. Will not change the output file on subsequent runs has reasonable-sized diffs when files changed
4. Capable of doing the work via //go:generate instead of forcing you to manually write additional Go code

The point #2 was important for me and the rest of the packages for one reason or another didn't work out that well.

From esc's README:

esc embeds files into go programs and provides http.FileSystem interfaces to them.

It adds all named files or files recursively under named directories at the path specified. The output file provides an http.FileSystem interface with zero dependencies on packages outside the standard library.

Solution 4:

You can use a string literal to define the text as a constant or variable. String literals are defined by enclosing the string with back-quotes. e.g. `string`.

For example:

package main

import "fmt"

func main() {
    const text = `
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec a diam lectus. Sed sit  
amet ipsum mauris. Maecenas congue ligula ac quam viverra nec consectetur ante 
hendrerit. Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet 
vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut 
libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, 
consectetur adipiscing elit. Aenean ut gravida lorem. Ut turpis felis, pulvinar a 
semper sed, adipiscing id dolor. Pellentesque auctor nisi id magna consequat sagittis. 
Curabitur dapibus enim sit amet elit pharetra tincidunt feugiat nisl imperdiet. Ut 
convallis libero in urna ultrices accumsan. Donec sed odio eros. Donec viverra mi quis 
quam pulvinar at malesuada arcu rhoncus. Cum sociis natoque penatibus et magnis dis 
parturient montes, nascetur ridiculus mus. In rutrum accumsan ultricies. Mauris vitae 
nisi at sem facilisis semper ac in est.
`

    fmt.Println(text)
}

Solution 5:

check packr, its quite friendly to use

package main

import (
  "net/http"

  "github.com/gobuffalo/packr"
)

func main() {
  box := packr.NewBox("./templates")

  http.Handle("/", http.FileServer(box))
  http.ListenAndServe(":3000", nil)
}