How to return a complex JSON response with Node.js?

Using nodejs and express, I'd like to return one or multiple objects (array) using JSON. In the code below I output one JSON object at a time. It works but this isn't exactly what I want. The response produced isn't a valid JSON response since I have many objects.

I am well aware that I could simply add all objects to an array and return that specific array in res.end. However I am afraid this could become heavy to process and memory intensive.

What is the proper way to acheive this with nodejs? Is query.each the right method to call?

app.get('/users/:email/messages/unread', function(req, res, next) {
    var query = MessageInfo
        .find({ $and: [ { 'email': req.params.email }, { 'hasBeenRead': false } ] });

    res.writeHead(200, { 'Content-Type': 'application/json' });   
    query.each(function(err, msg) {
        if (msg) { 
            res.write(JSON.stringify({ msgId: msg.fileName }));
        } else {
            res.end();
        }
    });
});

Solution 1:

On express 3 you can use directly res.json({foo:bar})

res.json({ msgId: msg.fileName })

See the documentation

Solution 2:

I don't know if this is really any different, but rather than iterate over the query cursor, you could do something like this:

query.exec(function (err, results){
  if (err) res.writeHead(500, err.message)
  else if (!results.length) res.writeHead(404);
  else {
    res.writeHead(200, { 'Content-Type': 'application/json' });
    res.write(JSON.stringify(results.map(function (msg){ return {msgId: msg.fileName}; })));
  }
  res.end();
});

Solution 3:

[Edit] After reviewing the Mongoose documentation, it looks like you can send each query result as a separate chunk; the web server uses chunked transfer encoding by default so all you have to do is wrap an array around the items to make it a valid JSON object.

Roughly (untested):

app.get('/users/:email/messages/unread', function(req, res, next) {
  var firstItem=true, query=MessageInfo.find(/*...*/);
  res.writeHead(200, {'Content-Type': 'application/json'});
  query.each(function(docs) {
    // Start the JSON array or separate the next element.
    res.write(firstItem ? (firstItem=false,'[') : ',');
    res.write(JSON.stringify({ msgId: msg.fileName }));
  });
  res.end(']'); // End the JSON array and response.
});

Alternatively, as you mention, you can simply send the array contents as-is. In this case the response body will be buffered and sent immediately, which may consume a large amount of additional memory (above what is required to store the results themselves) for large result sets. For example:

// ...
var query = MessageInfo.find(/*...*/);
res.writeHead(200, {'Content-Type': 'application/json'});
res.end(JSON.stringify(query.map(function(x){ return x.fileName })));