NServiceBus and Rabbit MQ or Kafka

Solution 1:

Years ago, I asked myself the same question. I was looking at NServiceBus to work with a different message queue, but the question was the same.

I decided not to use NServiceBus.

6 Month later, I realized I had re-built half of what NServiceBus did... only much more poorly.

The equivalent question of why would you need NServiceBus with RabbitMQ, is to ask why you would need the .NET Framework with ASP.NET MVC, or WinForms, or XAML, or any of the built-in libraries that .NET ships with, when you have the Common Language Runtime.

Shouldn't the CLR be enough, after all?

Of course not. Having the runtime on which code can execute - the MSIL interpreter and execution engine - is not nearly enough to be productive.

Sure, you can write command-line applications that take input and produce output. But try to build a real application without the common libraries - without the built-in SQL Server drivers; without any 3rd party controls or libraries. Build a Windows Desktop app without the System.Windows namespace.

You need those libraries to give you collections, and database access, and window objects and UI controls.

Similarly, RabbitMQ gives you everything you need to get started and working, but not enough to maintain productivity.

Sure, you can grab the .NET driver for RabbitMQ and start producing and consuming messages.

For a while, this will work just fine.

Pretty soon, you'll find yourself creating a wrapper around the driver, so you can reduce the amount of code you need to write.

Then you'll find yourself needing to deal with ack vs nack, and you'll create a simple API for that.

Then the need for dead-letter queues will pop up with nack calls, and you'll wrap that up in your API - simplified compared to the rabbitmq driver, of course.

Eventually, you'll want to deal with poison messages - messages that are malformed and causing exceptions. Once again, you don't want to write one-off code for this, so you'll write a library to handle it.

The list goes on and on.

6 months from now, you'll find yourself working with a half-written, barely specified, untestable library that only mimics the value and capabilities of NServiceBus (or MassTransit or whatever other service bus library you choose).

I won't say you have to use NServiceBus. And I would say you should learn how RabbitMQ works, without it. But once you get beyond the basics of sending and receiving messages, the value of NServiceBus and other service bus implementations, becomes very apparent, very quickly.

Solution 2:

It seems there is community support for Kafka transport in NServiceBus now: https://docs.particular.net/nservicebus/kafka/ (haven't tried it myself yet).