Node.js vs .Net performance

I've read a lot about Node.js being fast and able to accommodate large amounts of load. Does anyone have any real-world evidence of this vs other frameworks, particularly .Net? Most of the articles I've read are anecdotal or don't have comparisons to .Net.

Solution 1:

Being FAST and handling lots of LOAD are two different things. A server that's really FAST at serving one request per second might totally croak if you send it 500 requests per second (under LOAD).

You also have to consider static (and cached) vs dynamic pages. If you're worried about static pages, then IIS is probably going to beat node because IIS uses kernel-mode caching, which means that requests which request a static page are not even going to get out of the kernel.

I'm guessing that you're looking for a comparison between ASP.NET and node. In this battle, after everything's been compiled/interpreted you're probably going to be pretty close in performance. Maybe .NET's a little FASTER or maybe node's a little FASTER, but it's probably close enough that you don't care. I'd bet on .NET, but I don't know for sure.

The place that node is really compelling is for handling LOAD. This is where the technologies really differ. ASP.NET dedicates a thread for each request from its thread pool, and once ASP.NET has exhausted all the available threads requests begin to get queued up. If you're serving "Hello World" apps like the example by @shankar, then this might not matter that much because the threads aren't going to be blocked and you're going to be able to handle a lot of requests before you run out of threads. The problem with the ASP.NET model comes when you start making I/O requests that block the thread (call to a DB, make an http request to a service, read a file from disk). These blocking requests mean that your valuable thread from the thread pool is doing nothing. The more blocking you have, the less LOAD your ASP.NET app is going to be able to serve.

To prevent this blocking, you use I/O completion ports which don't require holding a thread while you wait for a response. ASP.NET supports this, but unfortunately many of the common frameworks/libraries in .NET DON'T. For example, ADO.NET supports I/O completion ports, but Entity Framework doesn't use them. So you can build an ASP.NET app that's purely asynchronous and handles lots of load, but most people don't because it isn't as easy as building one that's synchronous, and you might not be able to use some of your favorite parts of the framework (like linq to entities) if you do.

The problem is that ASP.NET (and the .NET Framework) were created to be un-opinionated about asynchronous I/O. .NET doesn't care if you write synchronous or asynchronous code, so it's up to the developer to make this decision. Part of this is because threading and programming with asynchronous operations was thought to be "hard", and .NET wanted to make everyone happy (noobs and experts). It got even harder because .NET ended up with 3-4 different patterns for doing async. .NET 4.5 is trying to go back and retrofit the .NET framework to have an opinionated model around async IO, but it may be a while until the frameworks you care about actually support it.

The designers of node on the other hand, made an opinionated choice that ALL I/O should be async. Because of this decision, node designers were also able to make the decision that each instance of node would be single threaded to minimize thread switching, and that one thread would just execute code that had been queued. That might be a new request, it might be the callback from a DB request, it might be the callback from a http rest request you made. Node tries to maximize CPU efficiency by eliminating thread context switches. Because node made this opinionated choice that ALL I/O is asynchronous, that also means that all it's frameworks/add-ons support this choice. It's easier to write apps that are 100% async in node (because node forces you to write apps that are async).

Again, I don't have any hard numbers to prove one way or another, but I think node would win the LOAD competition for the typical web app. A highly optimized (100% async) .NET app might give the equivalent node.js app a run for it's money, but if you took an average of all the .NET and all the node apps out there, on average node probably handles more LOAD.

Hope that helps.

Solution 2:

I did a rudimentary performance test between nodejs and IIS. IIS is about 2.5 times faster than nodejs when dishing out "hello, world!". code below.

my hardware: Dell Latitude E6510, Core i5 (dual core), 8 GB RAM, Windows 7 Enterprise 64 bit OS

node server

runs at http://localhost:9090/
/// <reference path="node-vsdoc.js" />
var http = require("http");
http.createServer(function (request, response) {
response.writeHead(200, { "Content-Type": "text/html" });
response.write("<p>hello, world!</p>");
response.end();
}).listen(9090);

default.htm

hosted by iis at http://localhost/test/
<p>hello, world!</p>

my own benchmark program using task parallel library:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Net;
using System.Threading;
using System.Threading.Tasks;
using System.Diagnostics;

namespace HttpBench
{
class Program
{
    private int TotalCount = 100000;
    private int ConcurrentThreads = 1000;
    private int failedCount;
    private int totalBytes;
    private int totalTime;
    private int completedCount;
    private static object lockObj = new object();

    /// <summary>
    /// main entry point
    /// </summary>
    static void Main(string[] args)
    {
        Program p = new Program();
        p.Run(args);
    }

    /// <summary>
    /// actual execution
    /// </summary>
    private void Run(string[] args)
    {
        // check command line
        if (args.Length == 0)
        {
            this.PrintUsage();
            return;
        }
        if (args[0] == "/?" || args[0] == "/h")
        {
            this.PrintUsage();
            return;
        }

        // use parallel library, download data
        ParallelOptions options = new ParallelOptions();
        options.MaxDegreeOfParallelism = this.ConcurrentThreads;
        int start = Environment.TickCount;
        Parallel.For(0, this.TotalCount, options, i =>
            {
                this.DownloadUrl(i, args[0]);
            }
        );
        int end = Environment.TickCount;

        // print results
        this.Print("Total requests sent: {0}", true, this.TotalCount);
        this.Print("Concurrent threads: {0}", true, this.ConcurrentThreads);
        this.Print("Total completed requests: {0}", true, this.completedCount);
        this.Print("Failed requests: {0}", true, this.failedCount);
        this.Print("Sum total of thread times (seconds): {0}", true, this.totalTime / 1000);
        this.Print("Total time taken by this program (seconds): {0}", true, (end - start) / 1000);
        this.Print("Total bytes: {0}", true, this.totalBytes);
    }

    /// <summary>
    /// download data from the given url
    /// </summary>
    private void DownloadUrl(int index, string url)
    {
        using (WebClient client = new WebClient())
        {
            try
            {
                int start = Environment.TickCount;
                byte[] data = client.DownloadData(url);
                int end = Environment.TickCount;
                lock (lockObj)
                {
                    this.totalTime = this.totalTime + (end - start);
                    if (data != null)
                    {
                        this.totalBytes = this.totalBytes + data.Length;
                    }
                }
            }
            catch
            {
                lock (lockObj) { this.failedCount++; }
            }
            lock (lockObj)
            {
                this.completedCount++;
                if (this.completedCount % 10000 == 0)
                {
                    this.Print("Completed {0} requests.", true, this.completedCount);
                }
            }
        }
    }

    /// <summary>
    /// print usage of this program
    /// </summary>
    private void PrintUsage()
    {
        this.Print("usage: httpbench [options] <url>");
    }

    /// <summary>
    /// print exception message to console
    /// </summary>
    private void PrintError(string msg, Exception ex = null, params object[] args)
    {
        StringBuilder sb = new System.Text.StringBuilder();
        sb.Append("Error: ");
        sb.AppendFormat(msg, args);
        if (ex != null)
        {
            sb.Append("Exception: ");
            sb.Append(ex.Message);
        }
        this.Print(sb.ToString());
    }

    /// <summary>
    /// print to console
    /// </summary>
    private void Print(string msg, bool isLine = true, params object[] args)
    {
        if (isLine)
        {
            Console.WriteLine(msg, args);
        }
        else
        {
            Console.Write(msg, args);
        }
    }

}
}

and results:

IIS: httpbench.exe http://localhost/test

Completed 10000 requests.
Completed 20000 requests.
Completed 30000 requests.
Completed 40000 requests.
Completed 50000 requests.
Completed 60000 requests.
Completed 70000 requests.
Completed 80000 requests.
Completed 90000 requests.
Completed 100000 requests.
Total requests sent: 100000
Concurrent threads: 1000
Total completed requests: 100000
Failed requests: 0
Sum total of thread times (seconds): 97
Total time taken by this program (seconds): 16
Total bytes: 2000000

nodejs: httpbench.exe http://localhost:9090/

Completed 10000 requests.
Completed 20000 requests.
Completed 30000 requests.
Completed 40000 requests.
Completed 50000 requests.
Completed 60000 requests.
Completed 70000 requests.
Completed 80000 requests.
Completed 90000 requests.
Completed 100000 requests.
Total requests sent: 100000
Concurrent threads: 1000
Total completed requests: 100000
Failed requests: 0
Sum total of thread times (seconds): 234
Total time taken by this program (seconds): 27
Total bytes: 2000000

conclusion: IIS is faster than nodejs by about 2.5 times (on Windows). This is a very rudimentary test, and by no means conclusive. But I believe this is a good starting point. Nodejs is probably faster on other web servers, on other platforms, but on Windows IIS is the winner. Developers looking to convert their ASP.NET MVC to nodejs should pause and think twice before proceeding.

Updated (5/17/2012) Tomcat (on windows) seems to beat IIS hands down, about 3 times faster than IIS in dishing out static html.

tomcat

index.html at http://localhost:8080/test/
<p>hello, world!</p>

tomcat results

httpbench.exe http://localhost:8080/test/
Completed 10000 requests.
Completed 20000 requests.
Completed 30000 requests.
Completed 40000 requests.
Completed 50000 requests.
Completed 60000 requests.
Completed 70000 requests.
Completed 80000 requests.
Completed 90000 requests.
Completed 100000 requests.
Total requests sent: 100000
Concurrent threads: 1000
Total completed requests: 100000
Failed requests: 0
Sum total of thread times (seconds): 31
Total time taken by this program (seconds): 5
Total bytes: 2000000

updated conclusion: i ran the benchmark program multiple times. Tomcat appears to be the fastest server in dishing out STATIC HTML, ON WINDOWS.

Updated (5/18/2012) Previously I had 100,000 total requests with 10,000 concurrent requests. I increased it to 1,000,000 total requess and 100,000 concurrent requests. IIS comes out as the screaming winner, with Nodejs fairing the worst. I have tabularized the results below:

.

Solution 3:

NIO servers (Node.js etc) tend to be faster than BIO servers. (IIS etc). To back my claim, TechEmpower is a company specialized on web framework benchmarks. They are very open and have a standard way of testing all framewrks.

Round 9 tests are currently the latest (May 2014). There are many IIS flavors tested, but aspnet-stripped seems to be the fastest IIS variant.

Here are the results in responses per second (higher is better):

• JSON Serialization
  • nodejs: 228,887
  • aspnet-stripped: 105,272
• Single Query
  • nodejs-mysql: 88,597
  • aspnet-stripped-raw: 47,066
• Multiple Queries
  • nodejs-mysql: 8,878
  • aspnet-stripped-raw: 3,915
• Plain Text
  • nodejs: 289,578
  • aspnet-stripped: 109,136

In all cases, Node.js tends to be 2x+ faster than IIS.

Solution 4:

I have to agree with Marcus Granstrom the scenario is very important here.

To be honest it sounds like you’re making a high impact architectural decision. My advice would be to isolate the areas of concern and do a "bake off" between whatever stacks you are considering.

At the end of the day you are responsible for the decision and I don’t think the excuse "Some guy on Stackoverflow showed me an article that said it would be fine" Will cut it with your boss.