Difference between a Message Broker and an ESB
You can use a transformation broker without a service bus, and vice versa. In terms of specific products I don't think any one is purely one or the other because of the way each complements the other. Some products are stronger in the one area, other stronger in another. Perhaps a choice needs to be made based on which function best covers an individual problem.
A broker may have better built-in "lego blocks" for constructing a transformation chain than an ESB product does. A broker pressed into service as an ESB may be crushed under load and not scale well, or may lack robust journaling and tools for dealing with journals.
Some ESBs allow database updates to be rolled back and queues to be replayed into a corrected application once an egregious error in logic has been uncovered and fixed. I don't think most brokers integrate that level of transactional support. For this to work at all your "transactions" almost have to be business events (a sale, a renewal, a change of ownership, etc.) rather than something like RPCish "database updates."
Disclaimer: I am an IBM consultant and specialise in WebSphere ESB. This comment isn't left in any official capacity.
An ESB is more of an architectural pattern or concept than a product - broadly, a service-based way of engineering loose coupling. Its definition is fought over and not exactly set in stone. In general, an ESB is set of unrelated (in a technical sense) services - they expose interfaces, and they consume them from other services. Generally there isn't a hub and spoke architecture involved, although there can be.
IBM certainly markets both WebSphere Message Broker and WebSphere ESB as products that make it easy to build an ESB (along with the DataPower hardware appliance). They have different technological roots, but have some overlap in purpose. Also, that's not to say you can't build an ESB with lots of other things that aren't branded as an 'ESB product'.
That doesn't answer all your questions, but hopefully addresses the IBM part.
The difference between a Message Broker and an ESB (Enterprise Service Bus) is mainly the word 'bus'.
To me, a Message Broker is one (usally big) process that transforms data from one structure to another structure or modifies content.
An ESB is a message oriented middleware (MOM) plus additional services, one of which could be a Message Broker. So an ESB can include a Message Broker as one of it's components. A Bus consists of more than one processes, otherwise I wouldn't call it a 'bus'. The nature of a bus is that there are multiple components serving different tasks, each one communicating over a MOM and adhering to some form of 'common data format'. A bus would consist of: applications sending data to the MOM, database adapters, Message Brokers, MOM bridges, etc.
The separation is a bit gradual, but the biggest difference between a Message Broker architecture and a Bus is that of granularity. If your task is to integrate applications A, B, .., Z and a couple of databases, you can do this with one big Message Broker connecting each and everyone. Or with an ESB where multiple small components take over just little tasks. For example one adapter connects to A, another one to B (but they don't do transformation), then each one sends their stuff to one (or more) Message Broker, each of which should be kept as simple as possible - e.g. not having to know about the data model of 'A' or 'B'. A good ESB should have a common data definition on the bus, abstracting from the 'differentness' of individual applications.
TRANSFORMATION: an ESB doesn't help with transformation, unless it comes with a Message Broker. But each good ESB should include a Message Broker anyway. The Message Broker should be your bus's expert for transformations, but nothing else.
HORIZONTAL scaling: if you only have 3 things to connect (now and forever), it's probably not worth the effort to get a full-blown ESB. A Message Broker has the advantage of being just one big process. You can configure everything in there and have a central location for all your data mappings, filtering and routing.
But if you have 30 applications to connect, one Message Broker would probably come to grinding halt. Of course you can buy more instances, run things redundant, etc. but you should change your strategy to 'localize' jobs. Each application's adapter (could be one little Message Broker instance each) should be able to generate and/or receive an abstracted common data model (e.g XML with a shared XSD). There could also be a central Message Broker for transformation tasks, but that instance should be unaware of data model A or B. So an ESB should move the processing to the expert component instead of keeping everything in a central place.
I just read this article by Udi Dahan a few days ago, which might give you a more clear view of what I feel is one fundamental difference.
http://www.udidahan.com/2011/03/24/bus-and-broker-pubsub-differences
Quoting:
The rule that there can only be a single publisher for a given event type is one of the things that differentiates buses from brokers, though both obviously allow you to have multiple subscribers
...
Unfortunately, there are many broker-style technologies out there that are being marketed under the banner of the Enterprise Service Bus. While some products have the ability to be deployed in both a centralized and distributed fashion (sometimes called “federated” or “embedded” mode), many do not enforce the “single publishing endpoint per event-type” rule.
Without this constraint, it is just too easy to make mistakes.
Hope it helps.
An Enterprise Service Bus provides three key values to the Business :
- Context- or content- based routing of transactions;
- Transformation from one message domain or transport to another message domain or transport;
- many-to-many service connectivity.
ESBs provide loose coupling of services, allow services to be reconstituted into entirely different application contexts than when the services were first envisioned or developed, and promote reuse of applications without the need to recode applications. WebSphere Message Broker (or now is called IBM Integration Bus) is a prime example of an Enterprise Service Bus. For an example of simplicity of code that brings to bear great power in a few lines, you can view my post here : http://soabus.org/viewtopic.php?f=3&t=13 . The fundamental construct inside the IIB runtime is called the Logical Message Tree (LMT). Everything that the developer wants to do is some type of operation on the LMT. ESQL is the most efficient language a developer can use to perform these operations on the LMT, although many other languages are supported (for example, Java, PHP, Python, etc.) No other product comes close to the efficiency and ease of developing ESB applications than IBM Integration Bus since 90 percent of the coding of these applications is done by dragging and dropping nodes onto a pallet. That leaves only 10 percent of the coding to be done by the Message Flow developer. By the way, WebSphere ESB has been discontinued by IBM and many of the competing products to IBM Integration Bus have not seen any new development on them for several years now. A list of various ESB product offerings can be seen at soabus.org.