Write objects into file with Node.js

I've searched all over stackoverflow / google for this, but can't seem to figure it out.

I'm scraping social media links of a given URL page, and the function returns an object with a list of URLs.

When I try to write this data into a different file, it outputs to the file as [object Object] instead of the expected: [ 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'] as it does when I console.log() the results.

This is my sad attempt to read and write a file in Node, trying to read each line(the url) and input through a function call request(line, gotHTML):

fs.readFileSync('./urls.txt').toString().split('\n').forEach(function (line){
    console.log(line); 
    var obj = request(line, gotHTML); 
    console.log(obj); 
    fs.writeFileSync('./data.json', obj , 'utf-8'); 
});   

for reference -- the gotHTML function:

function gotHTML(err, resp, html){ 
    var social_ids = []; 

    if(err){
        return console.log(err); 
    } else if (resp.statusCode === 200){ 
        var parsedHTML = $.load(html); 

        parsedHTML('a').map(function(i, link){
            var href = $(link).attr('href');
            for(var i=0; i<socialurls.length; i++){
                if(socialurls[i].test(href) && social_ids.indexOf(href) < 0 ) {
                    social_ids.push(href); 
                }; 
            }; 
        })
    };

    return social_ids;
};

Solution 1:

Building on what deb2fast said I would also pass in a couple of extra parameters to JSON.stringify() to get it to pretty format:

fs.writeFileSync('./data.json', JSON.stringify(obj, null, 2) , 'utf-8');

The second param is an optional replacer function which you don't need in this case so null works.

The third param is the number of spaces to use for indentation. 2 and 4 seem to be popular choices.

Solution 2:

obj is an array in your example.

fs.writeFileSync(filename, data, [options]) requires either String or Buffer in the data parameter. see docs.

Try to write the array in a string format:

// writes 'https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks'
fs.writeFileSync('./data.json', obj.join(',') , 'utf-8'); 

Or:

// writes ['https://twitter.com/#!/101Cookbooks', 'http://www.facebook.com/101cookbooks']
var util = require('util');
fs.writeFileSync('./data.json', util.inspect(obj) , 'utf-8');

edit: The reason you see the array in your example is because node's implementation of console.log doesn't just call toString, it calls util.format see console.js source

Solution 3:

If you're geting [object object] then use JSON.stringify

fs.writeFile('./data.json', JSON.stringify(obj) , 'utf-8');

It worked for me.