Parsing multiple JSON objects in Go

Objects like the below can be parsed quite easily using the encoding/json package.

[ 
    {"something":"foo"},
    {"something-else":"bar"}
]

The trouble I am facing is when there are multiple dicts returned from the server like this :

{"something":"foo"}
{"something-else":"bar"}

This can't be parsed using the code below.

correct_format := strings.Replace(string(resp_body), "}{", "},{", -1)
json_output := "[" + correct_format + "]"

I am trying to parse Common Crawl data (see example).

How can I do this?

Assuming your input is really a series of valid JSON documents, use a json.Decoder to decode them:

package main

import (
    "encoding/json"
    "fmt"
    "io"
    "log"
    "strings"
)

var input = `
{"foo": "bar"}
{"foo": "baz"}
`

type Doc struct {
    Foo string
}

func main() {
    dec := json.NewDecoder(strings.NewReader(input))
    for {
        var doc Doc

        err := dec.Decode(&doc)
        if err == io.EOF {
            // all done
            break
        }
        if err != nil {
            log.Fatal(err)
        }

        fmt.Printf("%+v\n", doc)
    }
}

Playground: https://play.golang.org/p/ANx8MoMC0yq

If your input really is what you've shown in the question, that's not JSON and you have to write your own parser.

Seems like each line is its own json object.

You may get away with the following code which will structure this output into correct json:

package main

import (
    "fmt"
    "strings"
)

func main() {
    base := `{"trolo":"lolo"}
{"trolo2":"lolo2"}`

    delimited := strings.Replace(base, "\n", ",", -1)

    final := "[" + delimited + "]"
    fmt.Println(final)
}

You should be able to use encoding/json library on final now.

Parsing multiple JSON objects in Go

Related

Recent Posts