How to find and replace in linux on a large file?

Oh, hi;

So I've got this big json file and I'm trying to find and replace a few things but when I've used sed I get an error and I need a sed or awk string to find and replace it, without getting an error.

• prajwal

Current:

{"telephone_numbers":["1-

Future:

{"telephone_numbers":["+1-

This is the command i tried and the error i got.

sed -i 's/{"telephone_numbers":["1-/{"telephone_numbers":["+1-/g' output.json
sed: -e expression #1, char 56: unterminated `s' command

Solution 1:

In the sed command you used

s/{"telephone_numbers":["1-/{"telephone_numbers":["+1-/g

[ in the pattern is interpreted as the beginning of a character set (e.g. [abc] or [a-z]). There is no closing ] and the whole logic breaks.

But you don't want character sets in the first place, you want literal [ characters. Escape them with \.

Strictly you need to escape the first one and you may or may not escape the second one. This is because character sets belong to regular expressions. In s/pattern/replacement/ the pattern is parsed as a regular expression but the replacement is not. Your second [ is supposed to be in the replacement where unescaped [ is not special. The fixed command will be:

sed -i 's/{"telephone_numbers":\["1-/{"telephone_numbers":["+1-/g' output.json

Solution 2:

I found a .go script that does this.

package main

import (
        "fmt"
        "io/ioutil"
        "os"
        "path/filepath"
        "strings"
)

func visit(path string, fi os.FileInfo, err error) error {

        if err != nil {
                return err
        }

        if !!fi.IsDir() {
                return nil //
        }

        matched, err := filepath.Match("*.json", fi.Name())

        if err != nil {
                panic(err)
                return err
        }

        if matched {
                read, err := ioutil.ReadFile(path)
                if err != nil {
                        panic(err)
                }
                //fmt.Println(string(read))
                fmt.Println(path)

                newContents := strings.Replace(string(read), "1-", "+1-", -1)

                fmt.Println(newContents)

                err = ioutil.WriteFile(path, []byte(newContents), 0)
                if err != nil {
                        panic(err)
                }

        }

        return nil
}

func main() {
        err := filepath.Walk(".", visit)
        if err != nil {
                panic(err)
        }
}

Than build the file

go build main.go

Lets run the script

./main

This script works on almost any file size, and any file type.