Todd Biske: Outside the Box

It’s been a while since I posted something more infrastructure related. Since my original post on the convergence of infrastructure in the middle was reasonably popular, I thought I’d talk specifically about the ESB: Enterprise Service Bus. As always, I hope to present a pragmatic view that will help you make decisions on whether an ESB is right for you, rather than coming out with a blanket opinion that ESBs are good, bad, or otherwise.

From information I’ve read, there are at least five types of ESBs that exist today:

1. Formerly EAI, now ESB
2. Formerly BPM, now ESB
3. Formerly MOM/ORB, now ESB
4. The WS-* Enabling ESB
5. The ESB Gateway

Formerly EAI, now ESB

There’s no shortage of ESB products that people will claim are simply rebranding efforts of products formerly known as EAI tools. The biggest thing to consider with these is that they are developer tools. Their strengths are going to lie in their integration capabilities, typically in mapping between schemas and having a broad range of adapters for third party products. There’s no doubt that these products can save a lot of time when working with large commercial packages. At the same time, however, this would not be the approach I’d take for a load balancing or content-based routing solution. Those are typically operational concerns where we’d prefer to configure rather than code. Just because you have a graphical tool doesn’t mean it doesn’t require a developer to use it.

Formerly BPM, now ESB

Many ESBs are adding orchestration capabilities, a domain typically associated with BPM products. In fact, many BPM products are simply rebranding of EAI products, and there’s at least one I know of that is now also marketed as an ESB. There’s definitely a continuum here, and the theme is much the same. It’s a graphical modeling tool that is schema driven, possibly with built-in adapter technology, definitely with BPEL import/export, but still requires a developer to properly leverage it.

Both of these two categories are great if your problem set centers around orchestration and building new services in a more efficient manner. If you’re interested in sub-millisecond operations for security, routing, throttling, instrumentation, etc., recognize that these solutions are primarily targeted toward business processing and integration, not your typical “in-the-middle” non-functional concerns. It is certainly true that from a modeling perspective, the graphical representation of a processing pipeline is well-suited for the non-functional concerns, but it’s the execution performance that really matters.

Formerly MOM/ORB, now ESB

These products bring things much closer to non-functional world, although as more and more features are thrown at the ESB umbrella, they may start looking more like one of the first two approaches. In both cases, these products try to abstract away the underlying transport layer. In the case of MOM, all service communication is put onto some messaging backbone. The preferred model is to leverage agents/adapters on both endpoints (e.g. having a JMS client library for both the consumer and provider), potentially not requiring any centralized hub in the middle. The scalability of messaging systems certainly can’t be denied, however, the bigger concern is whether agents/adapters can be provided for all systems. All of the products will certainly have a fallback position of a gateway model via SOAP/HTTP or XML/HTTP, but you lose capabilities in doing so. For example, having endpoint agents can ensure reliable message delivery. If one endpoint doesn’t have it, you’ll only be reliable up to the gateway under control of the ESB. In other words, you’re only as good as your weakest link. One key factor in looking at this solution is the heterogeneity of your IT environment. The more varied systems you have, the greater challenge you have in finding an ESB that supports all of them. In an environment where performance is critical, these may be good options to investigate, knowing that a proprietary messaging backbone can yield excellent performance, versus trying to leverage something like WS-RM over HTTP. Once again, the operational model must be considered. If you need to change a contract between a consumer and a provider, the non-functional concerns are enforced at the endpoint adapters. These tools must have a model where those policies can be pushed out to the nodes, rather than requiring a developer to change some code or model and go through a development cycle.

The WS-* Enabling ESB

This category of product is the ESB that strives to allow an enterprise to have a WS-* based integration model. Unlike the MOM/ORB products, they probably only require the use of agents on the service provider side, essentially to provide some form of service enablement capability. In other words, a SOAP stack. While 5 years ago this may have been very important, most major platforms now provide SOAP stacks, and many of the large application vendors provide SOAP-based integration points. If you don’t have an enterprise application server, these may be worthwhile options to investigate, as you’ll need some form of SOAP stack. Unlike simply adding Axis on top of Tomcat, these options may provide the ability to have intercommunication between nodes, effectively like a clustered application server. If not apparent, however, these options are very developer focused. Service enablement is a development activity. Also, like the MOM/ORB solutions, these products can operate in a gateway mode, but you do lose capabilities. Like the BPM/EAI solutions, these products are focused on building services, so there’s a good chance that they may not perform as well for the “in-the-middle” capabilities typically associated with a gateway.

The ESB Gateway

Finally, there are ESB products that operate exclusively as a gateway. In some cases, this may be a hardware appliance, it could be software on commodity hardware, or it could be a software solution deployed as a stand-alone gateway. While the decision between a smart-network or smart-node approach is frequently a religious one, there are certainly scenarios where a gateway makes sense. Perimeter operations are the most common, but it’s also true that many organizations don’t employ clustering technologies in their application servers, but instead leverage front-end load balancers. If your needs focus exclusively on the “in-the-middle” capabilities, rather than on orchestration, service enablement, or integration, these products may provide the operational model (configure not code) and the performance you need. Unlike an EAI-rooted system, an appliance is typically not going to provide an integrate anything-to-anything model. Odds are that it will provide excellent performance along a smaller set of protocols, although there are integration appliances that can talk to quite a number of standards-based systems.

Final words

As I’ve stated before, the number one thing is to know what capabilities you need first, before ever looking at ESBs, application servers, gateways, appliances, web services management products, or anything else. You also need to know what the operational model is for those capabilities. Are you okay with everything being a development activity and going through a code release cycle, or are there things you want to be fully controlled by an operational staff that configures infrastructure according to a standard change management practice? There’s no right or wrong answer, it all depends on what you need. Orchestration may be the number one concern for some. Service-enablement of legacy systems to another. Another organization may need security and rate throttling at the perimeter. Hopefully this post will help you on your decision making process. If I missed some of the ESB models out there or if you disagree with this breakdown, please comment or trackback. This is all about trying to help end users better understand this somewhat nebulous space.

3 Responses to “To ESB or not to ESB”