How do I read the contents of a Node.js stream into a string variable?

I'm hacking on a Node program that uses smtp-protocol to capture SMTP emails and act on the mail data. The library provides the mail data as a stream, and I don't know how to get that into a string.

I'm currently writing it to stdout with stream.pipe(process.stdout, { end: false }), but as I said, I need the stream data in a string instead, which I can use once the stream has ended.

How do I collect all the data from a Node.js stream into a string?

Solution 1:

Another way would be to convert the stream to a promise (refer to the example below) and use then (or await) to assign the resolved value to a variable.

function streamToString (stream) {
  const chunks = [];
  return new Promise((resolve, reject) => {
    stream.on('data', (chunk) => chunks.push(Buffer.from(chunk)));
    stream.on('error', (err) => reject(err));
    stream.on('end', () => resolve(Buffer.concat(chunks).toString('utf8')));
  })
}

const result = await streamToString(stream)

Solution 2:

None of the above worked for me. I needed to use the Buffer object:

  const chunks = [];

  readStream.on("data", function (chunk) {
    chunks.push(chunk);
  });

  // Send the buffer or you can put it into a var
  readStream.on("end", function () {
    res.send(Buffer.concat(chunks));
  });

Solution 3:

Hope this is more useful than the above answer:

var string = '';
stream.on('data',function(data){
  string += data.toString();
  console.log('stream data ' + part);
});

stream.on('end',function(){
  console.log('final output ' + string);
});

Note that string concatenation is not the most efficient way to collect the string parts, but it is used for simplicity (and perhaps your code does not care about efficiency).

Also, this code may produce unpredictable failures for non-ASCII text (it assumes that every character fits in a byte), but perhaps you do not care about that, either.

Solution 4:

(This answer is from years ago, when it was the best answer. There is now a better answer below this. I haven't kept up with node.js, and I cannot delete this answer because it is marked "correct on this question". If you are thinking of down clicking, what do you want me to do?)

The key is to use the data and end events of a Readable Stream. Listen to these events:

stream.on('data', (chunk) => { ... });
stream.on('end', () => { ... });

When you receive the data event, add the new chunk of data to a Buffer created to collect the data.

When you receive the end event, convert the completed Buffer into a string, if necessary. Then do what you need to do with it.

Solution 5:

What do you think about this ?

async function streamToString(stream) {
    // lets have a ReadableStream as a stream variable
    const chunks = [];

    for await (const chunk of stream) {
        chunks.push(Buffer.from(chunk));
    }

    return Buffer.concat(chunks).toString("utf-8");
}