How to efficiently concatenate strings in go

In Go, a string is a primitive type, which means it is read-only, and every manipulation of it will create a new string.

So if I want to concatenate strings many times without knowing the length of the resulting string, what's the best way to do it?

The naive way would be:

var s string
for i := 0; i < 1000; i++ {
    s += getShortStringFromSomewhere()
}
return s

but that does not seem very efficient.

New Way:

From Go 1.10 there is a strings.Builder type, please take a look at this answer for more detail.

Old Way:

Use the bytes package. It has a Buffer type which implements io.Writer.

package main

import (
    "bytes"
    "fmt"
)

func main() {
    var buffer bytes.Buffer

    for i := 0; i < 1000; i++ {
        buffer.WriteString("a")
    }

    fmt.Println(buffer.String())
}

This does it in O(n) time.

In Go 1.10+ there is strings.Builder, here.

A Builder is used to efficiently build a string using Write methods. It minimizes memory copying. The zero value is ready to use.

Example

It's almost the same with bytes.Buffer.

package main

import (
    "strings"
    "fmt"
)

func main() {
    // ZERO-VALUE:
    //
    // It's ready to use from the get-go.
    // You don't need to initialize it.
    var sb strings.Builder

    for i := 0; i < 1000; i++ {
        sb.WriteString("a")
    }

    fmt.Println(sb.String())
}

Click to see this on the playground.

Supported Interfaces

StringBuilder's methods are being implemented with the existing interfaces in mind. So that you can switch to the new Builder type easily in your code.

• Grow(int) -> bytes.Buffer#Grow
• Len() int -> bytes.Buffer#Len
• Reset() -> bytes.Buffer#Reset
• String() string -> fmt.Stringer
• Write([]byte) (int, error) -> io.Writer
• WriteByte(byte) error -> io.ByteWriter
• WriteRune(rune) (int, error) -> bufio.Writer#WriteRune - bytes.Buffer#WriteRune
• WriteString(string) (int, error) -> io.stringWriter

Differences from bytes.Buffer

• It can only grow or reset.

• It has a copyCheck mechanism built-in that prevents accidentially copying it:

  func (b *Builder) copyCheck() { ... }

• In bytes.Buffer, one can access the underlying bytes like this: (*Buffer).Bytes().

  • strings.Builder prevents this problem.
  • Sometimes, this is not a problem though and desired instead.
  • For example: For the peeking behavior when the bytes are passed to an io.Reader etc.

• bytes.Buffer.Reset() rewinds and reuses the underlying buffer whereas the strings.Builder.Reset() does not, it detaches the buffer.

Note

• Do not copy a StringBuilder value as it caches the underlying data.
• If you want to share a StringBuilder value, use a pointer to it.

Check out its source code for more details, here.