Todd Biske: Outside the Box

Archive for the ‘SOA’ Category

Ok, so Brenda Michelson called me out today on a conference call that I hadn’t made any comments regarding the impact of the economy on SOA initiatives. Of course, that’s like dangling a carrot in front of my face, so I now feel obligated to at least say something.

As I’ve stated before, I consider myself to be very pragmatic. I try to avoid knee-jerk reactions to highs or low. So, the recent buzz around how the economy would impact SOA efforts was a bit of a non-event for me. Companies have always had to deal with their own performance, and adjust their budgets accordingly, and obviously, the general economy has a role in that performance. Depending on your industry, it could be large, or it could be insignificant. So, whether it’s an SOA project or any other IT project, the decision making process is still the same. The only things that have changed are the resources that are available. If we have resources to do it, do it. If you don’t, it gets put off. Now, if we’re discussing how SOA initiatives should be prioritized relative to other IT projects, it’s a different debate, and one that is independent of the current funding at hand.

The other angle on this discussion is whether or not successful SOA adoptions will help companies when their revenue streams are not what they’ve been in the past. If we come back to the marketing terms for SOA: reuse, agility, etc., I think these are all realized in the form of IT productivity and efficiency. In other words, I’m either getting more things done in the some standard time frame, or I’m reducing the time it takes to get any particular solution done in comparison to similar efforts in the past. I firmly believe that SOA should be doing this, and therefore, it’s a logical conclusion that companies that have successfully adopted SOA are in a better position to execute in an economic downturn than their competitors that haven’t. Of course, that assumes that information technology plays a key role in a business’ ability to execute and can be a competitive differentiator. There you go, Brenda!

Ok, the events page is finally functional. I gave up on WordPress plugins, and am now leveraging an embedded (iframe) Google Calendar. A big thank you to Sandy Kemsley, as she already had a BPM calendar which I’m now leveraging and adding in SOA and EA events. I hope this helps people to find out the latest on SOA, EA, and BPM events.

I’ve decided to conduct an experiment. In the past, I’ve been contacted about posting information regarding events on SOA, either via a comment to an existing post, or through an email request. I’ve passed on these, but at the same time, I do like the fact that I’ve built up a reader base and my goal has always been to increase the knowledge of others on SOA and related topics.

What I’ve decided to try is having one post per month dedicated to SOA events that will be occuring in the near future, in addition to this permanent page on the blog that people can review when they please. I’m not about to go culling the web for all events, but I will collect all events that I receive in email at soaevents at biske dot com, and post a summary of them including dates, topic, discount code, and link to the detail/registration page. Readers would be free to post additional events in comments, however, I don’t think very many people subscribe to the comment feed, so the exposure would be less than having it in my actual post. I’m going to try to leverage a Google Calendar (iCal, HTML) for this, which will also be publicly available, even if someone wants to include it their own blog.

While this is essentially free marketing, in reality, I’d make a few more cents by having more visitors to this site than I would if all of these events organizers threw their ads into the big Google Ad Pool with the hopes of it actually showing up on my site. If I get a decent number of events from multiple sources the first two months, I’ll probably keep it up. If I only get events from one source, I’ll probably stop, as it will begin to look like I’m doing marketing just for them and I don’t want anyone to have the perception that my content is influenced by others.

The doors are open. Send your SOA, EA, and BPM events to me at soaevents at biske dot com. Include:

• Date/Time
• Subject/Title
• Description (keep it to one or two sentences)
• URL for more detail/registration
• Discount code, if applicable

If you want to simply send an iCal event, that would probably work as well, and it would make it very easy for me to move it into Google Calendar. My first post will be on Feb. 1, the next on March 1, and so on. I will post events for the current month and the next month, and may include April depending on how many events I receive.

Wow, I was very surprised to come into work to find out that Oracle had agreed to acquire BEA and that Sun had agreed to acquire MySQL. Based on the reaction of many in the blogosophere, like Miko Matsumura whose Facebook status said he was “amazed by sun buying MySQL and yawning at Oracle buying BEA” not many people were surprised by the Oracle/BEA deal. I actually was a little bit surprised, simply because it felt like BEA would stick it out as an independent, and it didn’t appear that Oracle was taking its “we will get what we want no matter how long it takes” approach as it did with PeopleSoft. That’s about all the comment I’ll make on this as I try to avoid too much speculation in the vendor space in this blog, but this was too big of an event to not mention. While I chose not to do 2008 predictions this year, all of those people who picked the no-brainer of more vendor acquisitions in the SOA space can check this one off your list only 16 days into the year.

The other acquisition was one that was very surprising in so much as not many people probably saw it coming. While not in the SOA space, it certainly shows that open source is becoming more and more ingrained in Sun. This is one that’s going to take some time to digest in the blogosphere to see what people will think about it. I think being associated with a bigger name can only help MySQL. As for the benefits to Sun, we’ll see. I’ll be interested to see what others have to say about it.

One thing that I have noticed over the past few years of my career is that no two companies define the role of the architect the same way. Personally, I view this as neither good nor bad, and this isn’t going to be a post on the whole architect certification efforts that exist out there. While it may make it more challenging to find qualified candidates when the definition of the role changes from company to company, once you’re in a company, clarity of the specific responsibilities at that company are certainly more important than whether or not you’re doing the same work as Joe or Jane Architect down the street. As another aside, I also think that those job responsibilities should serve as a guide, but not a barrier. All organizations require some flexibility in what people do if they expect to get things done versus devolving into a finger-pointing episode of people saying, “That’s not my responsibility.” Which brings us to the first type of role.

The Gluer

In this role, the person with the lucky fortune to have the architect title is expecting to hold everything together from a technical perspective. A typical software development project involves far more technical tasks than just writing code. Servers may need to be built, software installed, routers configured, etc. Put simply, the person playing this role was expected to be the glue that holds everything together from a technical perspective. You’re probably asking, “Isn’t this the job of the technical lead?” Well, the problem with the technical lead is that in most organizations I’ve seen, it was purely a role, and not a job title. Therefore, it was difficult to actually know who the technical leads were in an organization, get coverage across all projects, etc. Titles like senior developer or lead developer don’t cut it, because the emphasis is still on development, and not the other tasks involved with getting something into production. Unfortunately, giving them the title of architect is problematic. While it does allow an organization to establish some clear technical leadership, it’s likely that the individuals with this title will be so consumed with tactical project decisions, that very little of their time will actually be spent on architecture.

The Scheduler

While I never would have guessed this one, I heard it mentioned at two different organizations that architects are supposed to act as project managers for technical activities, normally working in conjunction with the “real” project manager. While I do think that all leaders should have some ability to do basic project management activities. That is, break down a goal into some set of constituent tasks on a timeline. There are certainly ties back to the gluer role, as this essentially takes the technical leadership a step further. In addition to identifying all of the technical activities, the person now has to manage all of them, rather than delegating this back to the project manager. Of all the roles, this one is the least desirable to me because I don’t consider project management a strong point for me. I’m admittedly a big picture thinker. Most good project managers I know are extremely detail-oriented. The best working scenario for me personally, is not to have me be the project manager, but work side-by-side with the project manager.

You’ll notice that both of these roles are very project-specific. If all of the architects in your organization are on project assignments, that’s a problem. Project, or solution, architecture is important, but you also need architects outside of projects, hence the next two roles.

The Decision Maker

This architect is normally not assigned to projects, but is still involved with the decision making process within projects. This person is likely viewed as the top of the technical hierarchy within some organizational or technical domain. If it’s associated with software development, I typically see it following organizational boundaries (e.g. an architect for all solutions developed by one development organizationy), while on some of the other activities, it follows technical domains, such as a Security Technology Architect, a Networking and Communications Technology Architect, etc. Projects go to these architects when decisions need to be made or approved. The challenge I’ve seen with this role is the flood gate, especially when the title is first established. Many organizations don’t have a technical hierarchy in their organization, and as a result, it can be unclear as to who has the technical decision making responsibilities. As a result, when someone is granted the title, the flood gates open, and every technical decision, even those that should be no brainers, wind up coming to those deemed “architects.” The challenge here is that the architect winds up having all their time consuming by decision making, with no time left over for establishing a strategy and direction.

The Strategist/Policy Maker

At the opposite end of the spectrum from the decision maker. Architects acting in this capacity are the legislative branch of the government, focused on established reference architectures, policies, roadmaps, etc. I thought about breaking this into two roles, because there are plenty of architects who don’t do strategy, but in general, the perception of the enterprise is similar. There’s a significant risk of becoming an ivory tower in this role. Just as the decision maker gets sucked back into project activities, the strategist can become disconnected from the reality of projects.

So what’s right? Personally, I think an organization needs gluers, decision makers, and strategists. We already have project managers, and as I previously stated, I don’t think we need to break out “technical” project management as a separate discipline. Should the remaining roles all have the “architect” title? In my mind, there’s really only debate about one role, the gluer. Clearly, not all of the activities associated with technical leadership have something to do with architecture. At the same time, however, if it’s not clear whose responsibility it is, these technical concerns will bubble their way up to the “decision makers.” If it’s necessary to bless that role with a title like “Solution Architect” to avoid this scenario, then do it.

What other roles and responsibilities have others seen with architecture? What’s missing from the list?

The IT as a service provider discussion was brought back up by Joe McKendrick of ZDNet in this blog. In it, Joe made reference to a past blog of mine in which I stated my opinion that a customer/supplier relationship between IT and their end users in the business was a bad thing, and I still believe that. Joe’s post brought to light some small nuances on that opinion that need clarification.

In my original post, I stated that IT moving solely to a supplier model to the business is an invitation to be outsourced. If you’re simply an order taker, there’s no reason that someone else can’t take those orders. The value-add that an internal IT department provides is not technology expertise, but technology expertise combined with knowledge of the company. Any SaaS provider or outsourcing agency must provide services to a mass market, or at least a market with more than one customer.

Getting back to Joe’s post, the inference that could be made is that IT shouldn’t be a service provider. This is not the case however. The IT-Business relationship should be a partnership, but you can’t be a partner if you’re not providing good service. Understanding the services that you bring to the table and doing them well is critical to the relationship. The difference is that those services do not define the boundaries of the relationship. Instead, they bring structure and foundation to it, on which partnership can be built. If your foundation is weak, the relationship will crumble. Therefore, adopting principles of service management within IT is a good thing, however, don’t approach it from the standpoint of competing against outside service providers. The decision to use outside providers should be made by the business, which includes IT. IT should be the one driving the discussion to say, “some aspects of our technology are really becoming commoditized and we can achieve some significant cost benefits through an external provider” rather than being told, “you’re a commodity and we’ve outsourced you.” In this sense, IT is no different than other business support organizations. Take HR as a good example. One could certainly argue that HR could be outsourced as it provides commodity services that all companies need. Every large company I’ve been at still has an HR department, though. What is more the norm is to have HR working as part of the business to make good business decisions on what aspects of HR to outsource, and what aspects of HR should remain within the company because there is value add. Only someone within the company can really understand the corporate culture which is critical to attracting and retaining talented individuals.

Be a partner in your business, but ensure that your partnership is on a solid foundation.

I received a couple comments on my previous post, and rather than respond in the comments themselves, I thought I’d use another blog post to address them since I don’t think many people subscribe to the comments feed. Before I get into them, I also wanted to call out this blog post from Anne Thomas Manes of the Burton Group. Out of a discussion on the Yahoo SOA Group about the relationship between 3-tier and SOA, she posted this entry which included this statement:

I also expect that the concept of “application” is likely to go away. Why is it that we as systems users have to be constrained by the limits of this artificial boundary called an application? Why do I have to shift my focus among multiple applications? Why do I have to copy data from one application to another? Why do I have to launch this stupid browser or that stupid application to get my work done? Why isn’t everything just accessible to me from my desktop (via widgets and gadgets) or from within my preferred operating context (e.g., email)?

It was this that prompted me to put together my original post, since I couldn’t find an entry in my blog that was specifically dedicated to this topic, although I had mentioned it as part of other entries. I had meant to include a link to Anne’s post in my first entry and forgot.

The first comment came from Brian “Bex” Huff who stated:

People understand the term “application,” but the word “solution” is a bit too nebulous.
Applications stand alone… what good is a widget to view an Excel doc, without the Excel application to create the doc in the first place?
I agree that IT should always *think* in terms of dynamic, evolving “solutions”… but the basic building blocks still include “applications”… as well as toolkits, frameworks, libraries, etc.

Bex actually made a statement that is indicative of the current culture, which was “Applications stand alone.” My opinion is that applications shouldn’t stand alone. Why shouldn’t we have the ability to present a UI component that can manipulate spreadsheets anywhere? Yes, there will always be cases where spreadsheet manipulation is the only thing we want to do, but there’s also plenty of cases where embedded spreadsheet manipulation would be better for users. What will enable this is thinking in terms of capabilities, rather than in terms of applications. My opinion is that an application is the result of packaging, rather than a unit of composition.

The second comment came from Rob Eamon. He wrote:

There will always be a collection of components/services that have been designed to work together to provide some set of functionality. And there will always be multiple sets of these. And there will always be a need for these sets to interact, possibly through mediation.
Disaggregating application components and making them independently accessible in many contexts seems very appealing.
But take the issues that the typical application development/support team faces and blow that up to the enterprise level. The fragile nature of such an approach will inherently stop it from becoming a reality.
IMO, the application is still a useful and reasonable building block and allows us to break down the enterprise solution space into manageable chunks.
Some of the application components might be useful as independent entities in a larger setting, but I donít think approaching everything that way is a wise course. IMO, it would inhibit flexibility rather than promote (counter-intuitive such that may be).

I wish Rob would have went into more detail on the “issues” that the typical application development/support team faces, because I can only guess what they may be. My first guess is that application teams currently have to deal with ever changing requirements and needs. If you increase the number of parts, you now have smaller components with new relationships associated with their use, and if we don’t manage it well, chaos will ensue. It should be noted that I’ve never said that this type of shift would be easy, and if anything I’d say it’s going to incredibly difficult. I’ve reflected on this in the past, specifically in this post, and wonder if it will take some company approach IT like this from the beginning, without any baggage to create the impetus to change.

Rob went on to state that in his opinion, the application is still a useful and reasonable building block. Here’s where I disagree, for the same reasons I used in Bex’s comments. An application is typically a “packaging” exercise, and those packages aren’t what we want for composition. The only part of an application that still has significant potential for being a “stand-alone” entity is the UI. I’d be happy to see an IT organization that makes an organizational/funding separation between development of UI code from development of services that are used by the presentation tier, much as Jeff Schneider suggested in this post from early last year.

Where I’ll agree with Rob is that this is not a change for the weak of heart. If a new CIO came in and reorganized the organization along these lines, the chaos that might ensue could certainly result in the perception that IT is even less agile than they were before. So, this isn’t a change that will occur in a year. This is a gradual, evolutionary change that will take time, but it will only happen if we’re committed to making it happen. I think a key to that is to get away from the application mindset.

Thanks to Rob & Bex for the great comments, and I’d love to hear from others whether you agree or disagree with my opinions.

I was very surprised that in ZapThink’s 2008 predictions, they predicted that they will be acquired:

Our prediction for 2008 is that one of these firms will acquire ZapThink, as well as other SOA thought leadership firms, because we can establish the winning acquirer as a global SOA leader

I wasn’t so much surprised that ZapThink would eventually be acquired, but I was surprised at the public announcement. Now, I’ve never been an entrepreneur, have never been involved with a startup, and have never gone through an acquisition, but a statement like this in an open forum sent out to ZapThink’s mailing list (beyond paying ZapThink clients) would seem to indicate one of two things:

1. An acquisition is already in the works.
2. The company is struggling and has just raised a huge banner saying, “We’re for sale. Please come save us.”

I’d be very surprised if it was the latter case, as a public statement like this would seem to kill any negotiation position. So, I’m going to assume that the former is the case and that it won’t be long before this “prediction” is reality.

It certainly brings up an interesting topic for companies in the SOA consulting space, one area where I do have past experience. Obviously, consulting companies make money by having billable resources. If you’re a consultant and you’re not billable, the revenue is not coming in to pay your salary. The owners of the company ultimately want your salary to be paid by clients, not by them. At the same time, this makes the development of intellectual property very difficult when the consultants are 100% billable. While intellectual property is certainly an indirect source of revenue, as it can help close deals, it’s not a direct source of revenue. So what’s a consulting firm to do? It would seem that bringing in a set of experts in intellectual property development that also have experience in creating non-consulting revenue streams (training, vendor marketing) could be a very potent combination that gets around the revenue challenge. The risk here, however, is that the “vendor neutrality” that an independent ZapThink has provided becomes challenging, given that many consulting firms get deals through their vendor partnerships, and it’s difficult to be a partner to competing vendors. Even ZapThink themselves have recognized the challenges of consulting and vendor relationships in Dave Linthicum’s recent post on his InfoWorld blog. We’ll just wait and see what the future holds for ZapThink. I wish my friends Ron, Jason, and Dave all the best in 2008.

Normally I’m not the guy to stir up a lot of controversy, but there’s one topic that I’ve mentioned on mailing lists, here, etc. that usually attracts a few opposing comments. That topic is the use of the term application in the enterprise. I’d like to see that term go away, or at least be limited to just the presentation layer of a solution. Why is that? It’s because it’s the boat anchor of IT, in my opinion. Think about it. How do we get out of the habit of thinking about monolithic point solutions for one small user group and into the habit of thinking about business capabilities that have potential for broader applicability when we have one hundred-plus organizations with the title “application development?” How do we deal with the organizational concerns when applications that were built for one group with one intent are now suddenly critical components for many other solutions with different user bases, each with their own set of priorities?

If nothing else, a switch from using the term “application” to the term “solution” would be a symbolic gesture that represents the change that must occur within IT. IT shouldn’t be producing applications, we should be producing solutions. Those solutions may still be self-contained like yesterday, or they may be a composite of pieces from many previous solutions, some new development, some off the shelf solutions, and some Internet-hosted offerings. The boundary of what constitutes “one application” are all but impossible to draw because of the many necessary interdependencies. Even if we limit the use of the term to just the presentation layer, it’s still debatable whether we should be using it. For example, are widgets on the OS X Dashboard applications? We don’t call them applications, we call them widgets. There’s some pretty sophisticated widgets, though. But wait a minute, won’t Word and Excel be around forever as applications? Well, the latest version of OS X introduced Quick Look, where with a press of the space bar, I can see the contents of an Excel spreadsheet without launching “an application.” The ability to present information in a tabular format is merely a capability. So, even in the sacred ground of the desktop, there are arguments that this notion of application launching may become less important to where the right thing simply happens.

I’m not a fool, and I certainly don’t think the use of the term application is going to go away in 2008, or anytime soon, but I do think that to continue the innovative use of technology, we need to make sure that the terms of the past aren’t locking us in to a single way of thinking about how computers are used. The continued need for better integration and more context-awareness in our technology solutions will only increase, and that pressure will continue to challenge any organization that is stuck with the application-centric view of technology as their boat anchor.

To all my readers, have a Merry Christmas and Happy New Year!

Last year, like many other pundits, I decided to have a go at some predictions for 2007. Let’s revisit them.

• Vendors: Surprise, surprise, the consolidation will continue. There aren’t too many niche SOA vendors left, and by the end of 2007 there will be fewer.
  Okay, so this was a no-brainer, but the devil is in the details. We had a very big deal in the SOA space between SoftwareAG and webMethods, and the deal that hasn’t happened between Oracle and BEA. There really wasn’t much that happened with the smaller SOA vendors (AmberPoint continues to chug along in their narrow niche), so maybe that category has already reached its threshold. There were far more dealings outside of the SOA space, clearly around business intelligence.
• Operational Management: At least one of the major Enterprise Systems Management providers will actually come out with a decent marketing effort around SOA management along with a product to back it up…
  My wishful thinking didn’t pan out. I’m still astonished at the lack of activity around this from the big players in Enterprise Systems Management. While I understand that management software is an extremely difficult sell, you’d think that a connection could be made into the visibility provided by an SOA management system and the growing interest and importance in business intelligence. Overall, the marketing message is still louder from companies like AmberPoint, SOA Software, and Progress Actional than from the big players.
• Registry/Repository: At least one players in the CMDB space will enter into the Registry/Repository space, most likely through acquisition. Thereís simply too much overlap for this not to occur.
  Once again, this didn’t pan out, however, in my conversations this past year I ran into far more people who now understand the relationship between CMDB and the SOA Registry/Repository, although I think some of the lower level marketing and sales people at the conferences need to educate themselves some more on this, because many of them didn’t get it. At best, they only understood a basic need to grab some list of services via UDDI from a registry.
• CMDB: At least one of the super-platform vendors will see the overlap between CMDB and Registry/Repository and begin to incorporate it into their offerings, either through partnership, or through an acquisition …
  I won’t be as harsh on myself on this one. While none of the super-platform vendors have done what I’ve said, the majority of them have begun to show the importance of the metadata repository in their overall infrastructure ecosystem. Whether it is Oslo, IBM’s WSRR, BEA’s ALRR, SoftwareAG’s CentraSite, or any of the others, expect to see more on this one.
• Events: … I think weíll see renewed interest in event description, as I see it as a critical tool for enabling agility. Services provide the functional aspect, but there needs to be a library of triggers for those functions. Those triggers are events. Along with this, the players in the Registry/Repository space will begin to market their ability to provide an event library just as they can provide a service library.
  I should have re-read my previous postings on the slow uptake of CEP and other event technologies, even though it stirred the pot with some CEP vendors. I haven’t seen much of anything on the event description front or any discussion of event libraries from the SOA Registry/Repository space.
• Service Development and Hosting: … While itís too early to proclaim the application server dead in 2007, I think the pendulum will begin to swing away from flexibility (i.e. general purpose servers that host things written in Java or C#) and toward specialization. A specialized service container for orchestration can provide a development environment targeted for orchestration with an execution engine targeted for that purpose …
  Thanks to Microsoft and Oslo, I think I can claim a winner on this one. The pace of adoption may be slower than the wording in my prediction, but I do think it’s safe to say that more companies are trying to leverage model-driven BPM-based technologies for development than last year. The key question is whether they’re seeing the success they desire or not. A subject for another blog entry is the current reality of model-driven development with today’s BPM tools.

Overall, I think my predictions are about typical of my thinking. I freely admit that I’m a forward thinker, and as a result, I’m usually guilty of thinking things will happen faster than they actually do. I haven’t decided whether or not to make 2008 predictions yet, and given that the bulk of my 2007 predictions are still off in the future, I don’t want to do a rehash of the same thing. We’ll see. If I feel inspired after seeing others publish their own predictions, perhaps I’ll do it again. One prediction I didn’t make was how many blog posts I’d have, and I was surprised to see that my 2007 predictions was post 103, and this post will be number 345. While it’s not as easy to find time to post now that I’m back in the corporate world, hopefully you found some of those 200+ posts enlightening and you’ll do the same in 2008. I’ve seen a pretty significant uptick in the number of subscribers over the year from FeedBurner, so I’m thankful for all of you continuing to read and sharing the links with others.

Just when you thought it was safe to build your SOA, the ESB has returned. A whitepaper from Paul Fremantle of WSO2 seems to have stirred the ESB pot once again, with a number of pundits chiming in on the future of the ESB. I was surprised at how little conversation there was about at the Gartner AADI Summit in comparison to the previous Gartner event I had attended two years prior, but that’s quickly changed.

I read Paul’s paper, and from my perspective, not much has changed from the views that I expressed two years ago back in a Burton Group Catalyst presentation and then in this followup blog post. The bulleted list of capabilities provided in that post aren’t ones that are going to go away, and it surprises me that there continues to be so much debate in this space.

Joe McKendrick recently summarized a number of recent comments in a blog entry, including ones from Roy Schulte of Gartner, Lief Davidson of IBM, ZDNet blogger Dana Gardner, and Lorraine Lawson of IT Business Edge (who has given a couple of my recent blogs some publicity, thanks for that). There were a couple things in Joe’s entry that just rubbed me the wrong way.

First is this notion of federated ESBs. Both Roy and Dana made comments around this, and I simply don’t agree. I’ll admit, there are a certain class of organizations that will have multiple ESBs. They’re the same organizations that have one of just about every technology because they’re so huge. Take them out of the equation, because I don’t believe they represent the masses. For the rest of us, what does “federated ESBs” mean? Does it mean that I have multiple ESBs performing redundant capabilities? Or does it mean that I partition up the capabilities across multiple products/devices, but yet with no two products providing the same capability, even though they may be capable of it? The latter is tolerable and likely, the former is not. For example, Roy mentioned security policies and quality of service. I may rely on an XML appliance for service security at the perimeter, and then rely on app server capabilities or some security agent within the data center. For simplicity sake, if we define quality of service as some form of intelligent routing and traffic shaping, I may rely on an ESB, a high-end XML appliance or network device, or advanced clustering capabilities of an app server. This type of partitioning makes sense. What doesn’t make sense is having one ESB (pick your favorite) providing security, QoS, etc. for services by development group A and having another ESB providing the same for services from development group B. To put it in context, routing is a core capability according to my definitions. Would you let development group A put a Cisco router in front of their services and development group B put a Juniper router in front of their services? You certainly wouldn’t do this if all of the services are hosted in the same data center. Yes, if your company has grown through acquisition and has lines of business all over the place, you may have different routers, but now we’re getting back to those super-large conglomerates I mentioned at the beginning. For those of you now envisioning the “big honking ESB,” don’t think that way. Think of it more like the network. I may have many deployments of a single ESB product, each handling a portion of the services in the enterprise, and their consumers know to direct to the appropriate ESB from design time rather than some uber-ESB. Applications point at specific databases or hostnames, there’s no reason that services can’t work the same way. Again, I don’t view this as federation.

What I believe we need to strive for in this space is centralized policy management and distributed enforcement, with standards based communication between the policy manager and the enforcement points (if only those standards existed). For example, one could make the argument that SAP could provide a second ESB that deals with services provided by the SAP infrastructure. I don’t view this as federation, however. From my perspective, I don’t even care whether SAP has an ESB or not. What I do care about is whether the entry points exposed by SAP can enforcement my QoS policies, my security policies, my versioning policies, etc. Let me centrally manage the policies, push them out, and have it just work. The challenge with this is not the enforcement architecture and federation there, but rather the metadata repository that will hold all of this policy information. This is where federation is important, because I may be getting third party products that come with their own pre-populated registry/repository of services that I need to manage.

The second statement that I disagreed with was Paul’s comment that ESBs discourage the shared ownership of services. If it does, then I think ESB ownership is the problem. Most web applications require the configuration of a load balancing farm before they can be accessed. No one considers the keeper of the farms in network operations the “owner” of those web applications, so why would the team responsible for the configuration of the ESB be considered the owner of services? Personally, I think a lot of this stems from ESBs being targeted at developers, rather than at operations teams. As I’ve commented in the past, I think the inclusion of service development capabilities like orchestration and the leftovers from the EAI space messed up the paradigm and caused confusion. Keep mediation separate from development.

So, as I climb down from my soap box, what are my parting words? I still believe that the capabilities in my post from last year are necessary, and an ESB is one way of providing those capabilities. Intermediaries are almost always used in web architectures, so there shouldn’t be such a strong aversion to having them as a front end to a service. That being said, too many intermediaries is a bad thing, because we haven’t gotten to the single pane of glass management that I think is necessary. Rather, each intermediary has its own management console, and the chances of something getting missed or fat-fingered goes up. Focus on an approach for centrally managing the policies, minimizing the number of places where policies are enforced, and keeping operational activities separated from service development activities.

James McGovern called out that he hasn’t seen much discussion on the topic of reference architectures, and called me out for my thoughts. I’m never one to pass up a good blog topic, especially when I don’t have to come up with on my own.

First, some background on my experience. My current job responsibilities include the development of reference architectures, my engagements with clients while I was a consultant all included the development of either a deliverable that included “reference architecture” in the title, or was clearly some form of reference material, and my job prior to consulting, included the development of reference architectures within the last 12 months of my time there. So, I’m no stranger to this space.

Reference architectures, and reference materials (since the need doesn’t stop at architecture) in general are an interesting beast. Personally, I view them as part of the overall governance process, mainly because they’re created to document a desirable pattern or approach that the authors would like (or will ensure that) others follow. At the same time, a document alone does not create governance, just as buying an SOA Registry/Repository doesn’t create SOA governance. Reference materials are a tool in the arsenal and the degree to which they are used is dependent on how you architects work with the end consumer of the reference material. Organizations are all over the spectrum on this. Some architects live in an ivory tower with virtually no interaction with teams on active projects, some architects are the exact opposite, with their time completely consumed by day-to-day project activities. Most organizations fall somewhere in between.

My opinion is that reference material is absolutely necessary, if nothing else but to prevent the organization from tribal operations. If none of the standards and guidelines ever get written down, and decisions are solely based on tribal knowlege, the organization can quickly break down into the haves and the have-nots. If you’re part of the tribe, you have the knowledge. If you’re not, all you can do is make your best guess until you have to show up to tribal council and get lambasted. Trying to gain the knowledge from the outside is a very difficult process.

The next question, however, is what information belongs in the reference material? Does it do any good to document something in the reference architecture that everyone should already know, or should you assume that no one knows anything, and document it all? The problem is that EA has limited resources, just like everyone else, so you have to give consideration to the bang for the buck in the reference material. Once again, what’s “right” is very dependent on the end consumer of the material (which is why having a consumer focus is important). If you have an organization of seasoned Java programmers, how much reference material is needed on developing good web applications? If you have an organization with lots of VB6 and COBOL developers, they may need lots of reference material on web applications. So, know your audience, and make sure that the reference material is relevant and valuable for them.

James Urquhart brought to my attention the public review of SOA Patterns, as authored in the forthcoming book, “SOA Design Patterns,” by Thomas Erl. You can see the press release from Prentice-Hall here.

My first reaction when I received the email, prior to visiting the SOAPatterns.org site was one of skepticism. While I think patterns can add a lot of value, the immediate problem I saw stems from the fact that I’m very much a believer in business-driven SOA. In order to reach a broader audience across multiple verticals, you have to be more business agnostic. As we get more business agnostic, we naturally move deeper into the technology stack, and things at that level of granularity may not be the best service candidates, although they may be great candidates for reusable frameworks. If we’re talking about patterns inside of the service implementation, then we’re really talking about general design patterns, building on the original work of the Gang of Four, not really SOA Design Patterns.

So, with my bias set, I visited the web site. The first thing I hoped to see was some classification by business industries, such as “SOA Patterns for Insurance” or “SOA Patterns for Health Care” but I didn’t find them. Bummer, but I also didn’t expect this. Something like that would be of significant value as intellectual property to a consulting firm, and I think they’d make a lot more money keeping it to themselves and leveraging it on their engagements. What was on the site was four chapters: Basic Service Inventory Design Pattern Language, Architectural Design Patterns, Basic Service Design Pattern Language, and Service Design Patterns.< ?)>

In looking at the first chapter, Basic Service Inventory Design Pattern Language, my first reaction was again one of skepticism. The first page begins with “Inventory Context Design Patterns,” “Inventory Boundary Design Patterns,” “Inventory Structure Design Patters,” and “Inventory Standardization Design Patterns.” It also introducted a phrase- “service inventory architecture” -which I had never heard before. Looking at this page, nothing was making a connection with me. As I drilled into each section, I did find some goodness, but it could be argued that what is really being presented in this section is really a description of a step in a methodology, rather than a pattern. For example, one pattern listed is the “Enterprise Inventory” pattern, which lists the problem as:

Delivering services independently via different project teams across an enterprise establishes a constant risk of producing inconsistent service and architecture implementations, compromising recomposition opportunities.

The solution is:

Services for multiple solutions can be designed for delivery within a standardized, enterprise-wide inventory architecture wherein they can be freely and repeatedly recomposed.

This doesn’t feel like a “pattern” to me, but it’s certainly something that should be done. I don’t think anyone would argue that having an enterprise service inventory is a bad thing. Another pattern I looked at was the “Vendor-Agnostic Context” pattern. Again, what was presented in the “pattern” was goodness, however, it felt more like a principle rather than a pattern. This particular one did do a good job in demonstrating how this principle does lead to the use of specific techniques, such as leveraging the “Canonical Protocol” and “Decoupled Contract” patterns.

Overall, what did I think? Well, it certainly didn’t meet my original hope of seeing industry-specific business patterns. By that, I mean I didn’t find something that said “Order to Cash” with guidance on the types of services that should make up that process. I didn’t find something that said, “here are services that all organizations with an HR department should have.” Nothing business-specific whatsoever. Of course, I didn’t expect to see this, it’s just what I was hoping to see, just as I hoped the (defunct?) SOA Blueprints effort from OASIS a couple years ago might produce something along these lines.

Putting that bias aside, the more I dug into the site, the more I found things that provided good guidance, even though I’d say the use of the “pattern” moniker was a bit liberal. If you want to get an idea of what principles and factors should be considered in creating good services, versus just slapping WSDL or XML schemas in front of some existing logic, there’s a lot of good material here that is freely available. While some of the earlier pages read too much like a college textbook, a couple drilldowns brought me to things that were applicable to my daily work and made sense. So, based on that, I would recommend that people at least visit the patterns site, drill down into it, and see what nuggets you can leverage in providence guidance to your service development teams. If you really like it, then perhaps Thomas’ book can become part of your standard library for your developers.

In a comment on my “EA and SOA Case Panel” entry, Surekha Durvasula asked me a couple questions. They didn’t come up in the panel discussion, so I thought I’d respond in a separate entry, as the topic should be of interest to many of my readers. She wrote:

Is “reuse” of a business service considered a valuable metric? How does governance influence the “reusability metric”? Did this come up during this SOA panel?

Specifically, I am wondering if service governance has any bearing in terms of not only promoting the usage of a service but also in ensuring that the enhancement of a service is in keeping with the enterprise-worthiness of the service. Often times it is the evolution of the service where cross-domain applicability is sacrificed.

Also, is there a trend in the industry in terms of promoting business service usage via the use of a “rewards program” or in tying it to compensation packages? Have some industries reached a level of maturity in terms of service reuse especially in those industry verticals that are hit with global competition forcing them to reduce overall operations costs and/or to offer novel product offerings?

Let’s take these one at time. On the subject of reuse, I absolutely think that number of consumers is a valuable metric. At the same time, when dealing with reuse, one must be cautious that it isn’t the only metric of interest. I’ve been in meetings with individuals that have made comments like, “If a service isn’t reused, why are you making it a service in the first place?” I strongly disagree with statements like this, as do most pundits in the SOA space. To defend this position, I frequently quote the oft-referenced Credit Suisse SOA efforts, where they stated that their average number of consumers per service was 1.5. This means that there will be many services that aren’t reused, and probably some that are used by many consumers. While reuse is important, we also have to be looking at metrics for agility, which loosely stated, is the ability to respond to business change. This will involve tracking the time it takes to develop solutions. The theory is that by breaking a solution apart into autonomous services, I reduce the number of touch points when the business needs change. In reality, it depends on the type of change. For example, most of us would agree that a separation of presentation logic from the core business processing is a good thing. That being said, there certainly are plenty of changes that will require touching both the presentation logic and the business logic. One of the most difficult parts of SOA is knowing where to draw service boundaries, because the rules are always changing.

Back to the subject- if we have reusable services, what role does governance play in ensuring that the service doesn’t fork into a bunch of one-off, consumer-specific variants? This is a very interesting question, one that I hadn’t thought much about in the past. My gut is telling me that the burden for this belongs with the service manager, not with a governance team. That’s not to say that there shouldn’t be any involvement from the governance group, but I see a much stronger role from governance in establishing the original service boundaries and assigning service ownership. For future versions, the service manager must be the one that can recognize when the service is getting outside of the boundaries that were originally intended, and this will happen. In some cases, boundaries may need to be redefined. In other cases, we may need to push back on the consumers. All of this starts with that service manager. The service manager must balance the needs of the consumer against the cost of service management. Measurements for determining that manager’s performance should include the number of versions currently being managed and the time required to respond to consumer requests. It is then in their best interests to keep the service focused on its original purpose.

Finally, regarding “rewards programs” or incentives, I don’t know that I’ve ever heard of a case study centered around reuse that didn’t involve incentives. SOA is about culture change, and it’s extremely difficult to change culture without incentives. One only need to look at a government to understand how change occurs. No one would be happy if the federal government mandated that all cars sold starting in 2008 had to get 50 mpg or higher. This is the “big stick” approach. I’ve got a big stick and you’ll get whacked with it if you don’t comply. In terms of IT incentives, one manager I worked with summed up the “big stick” approach well, “Your incentive is that you’ll keep your job.” More typically, the government takes a “carrot” approach, at least at the beginning. Tax breaks are granted to companies that produce high mpg vehicles and to consumers that buy them. These incentives may not even cover the added cost of that approach (e.g. does a $500 tax break for 4 years justify spending $3000 more on a vehicle?), but just the fact that they exist can often be enough to encourage the behavior. Only when enough momentum has gathered does the stick come out, essentially stating a policy that is what the majority of the people are doing already. Overall, I think that incentives should be viewed as a short-term tool to get momentum behind the change, but should always be planned for phase-out once the desired behavior is achieved. Have we reached that point with SOA? I’ve yet to see a company that has. Have we reached that point with reusable libraries? Partially. Most developers would not build their own low-level frameworks today. The problem, however, is that multiple frameworks exist, and there’s still strong resistance in many organizations to having a single solution coming from a frameworks team. I heard my first talk on reuse back in 1998, so it’s very clear that widespread culture change takes a long time to do.