What is an ESB and what is it good for?

It's a fairly high level concept of abstraction. The central concept is that the ESB provides the middleware and interfaces that allow businesses to connect their applications without writing code.

This could include mediation to reconcile incompatible protocols, data, and interaction.

The idea of a central bus on which everything passes gives opportunity for additional layers of abstraction. Using industry standards to "plug" other applications, clients, and such into this bus makes it so that connecting new services, data sources, clients with disparate needs is relatively easy.

Actual implementations

As far as actual implementations, that's the domain of very large enterprise support businesses. While it's very buzzwordy, the goal is an ideal that on some small level can be understood through comparison with the internet:

Similarity to Internet

One big communications bus with widely different uses and data, but all running standardize protocols.

One can, in fact, write an HTTP to FTP connector that would allow browsers to access FTP sites without invoking an FTP client (usually built into the browser now).

Mashups

Mashups demonstrate an interesting implementation - take some bus route data from the San Francisco authority, map from google, and sushi bar locations from yahoo with ratings and run a simple query that gives you the closest sushi bar, weighting it so that you'd be willing to travel a little further for a better bar.

All completely different services, incompatible by themselves, but using standard connectors (yahoo pipes, for instance) they can be pulled together into a cohesive and useful whole.

-Adam

Disclaimer: I work for IBM and consult on WebSphere ESB, an IBM product designed to build ESBs with. The following are my opinions and don't necessarily reflect IBM's position.

An ESB is different things to different people, unfortunately.

To me, an ESB is a any technology that you can insert into an SOA (Service-Oriented Architecture), allowing you to connect disparate systems together. It often performs the functions of protocol transformation, message modification, routing, logging, acting as a security gateway, and so on. For example, you might use an ESB to expose a service previously only available as a Web Service as a JMS-based service as well.

In this respect, ESB implementations (or to be more precise, software sold to build ESBs with - such as that I consult on) are often technologically similar to what used to be known as a messaging or queueing broker, although the purpose is somewhat different, because (as the acronyms imply) it's oriented around services rather than moving messages from one place to another. How important the distinction is technologically is a matter of opinion.

My experience with commercial ESB is it's an overblown and expensive technology that creates as many problems as it solves. The ESB will link new systems and legacy, messages will fly over the bus and everything will be able to talk to everything else seamlessly. Throw in some resilience, orchestration and you have a very powerful piece of enterprise application software.

The problem comes when you try to use them for real, the overhead of writing for the bus, creating the message structures and so on, can outweight the benefits. As a high cost item, the ESB is seen as the panacea for all tech issues which it isn't, too much time is spent on the bus and not on the applications/data being connected. It's often the case that multiple competing standards will fight for supremacy in the same organisation leading to the classic tech dominated silos that these systems should actually be fixing.

IMHO it is far better to use create a small number of specific interfaces, typically using web services between only those system that need it.