Todd Biske: Outside the Box

Archive for the ‘SOA’ Category

I received two questions in my email regarding my previous post on Service Lifecycle Management, specifically:

• Who within an organization would be a service manager?
• To whom would the service manager market services?

These are both excellent questions, and really hit at the heart of the culture change. If you look at the typical IT organization today, there may not be anyone that actually plays the role of a service manager. At its core, the service manager is a relationship manager- managing the interactions with all of the service consumers. What makes this interesting is when you think about exposing services externally. Using the concept of relationship management, it is very unlikely that the service manager would be someone from IT, rather, it’s probably someone from a business unit that “owns” the relationship with partners. IT is certainly involved, and it’s likely that technical details of the service interaction are left to the IT staff of each company, but the overall relationship is owned by the business. So, if we only consider internal services, does the natural tendency to keep service management within IT make sense? This approach has certain risks associated with it, because now IT is left to figure out the right direction through potentially competing requirements from multiple consumers, all the while having the respective business units breathing down their neck saying, “Where’s our solution?” At the same time, it’s also very unlikely that business is structured in such a way to support internal service management. Many people would say that IT is often better positioned to see the cross-cutting concerns of many of these elements. So, there are really two answers to the question. The first answer is someone. Not having a service owner is even more problematic than choosing someone from either IT or the business who may have a very difficult task ahead of them. The second answer is that the right person is going to vary by organization. I would expect that organizations whose SOA efforts are very IT driven, which I suspect is the vast lot of them, would pick someone within IT to be the service manager. I would expect that person to have an analyst and project management background, rather than a technical background. After all, this person needs to manage the consumer relationship and understand their requirements, but they also must plan the release schedule for service development. For organizations whose SOA efforts are driven jointly with the business, having a service manager within a business organization will probably make more sense, depending on the organizational structure. Also, don’t forget about the business of IT. There will be a class of services, typically in the infrastructure domains, such as authentication and authorization services, that will probably always be managed out of IT.

On question number two, I’m going to take a different approach to my answer. Clearly, I could just say, “Potential service consumers, of course” and provide no help at all. Why is that no help? Because we don’t know who represents those service consumers. Jumping on a common theme in this blog, most organizations are very project-driven, not service or product driven. When looking for potential service consumers, if everything is project driven, those consumers that don’t exist in the form of a project can’t be found! I don’t have a background in marketing, but I have to believe that there are probably some techniques from general product marketing that can applied within the halls of the business to properly identify the appropriate segment for a service. The real point that needs to be made is that a service manager can not take the field of dreams approach of simply building it, putting some information into the repository, and then hoping consumers find it. They have to hit the pavement and go talk to people. Talk to other IT managers whom you know use the same underlying data that your service does. Talk to your buddies at the lunch table. Build your network and get the word out. At a minimum, when a service is first identified, send a blast out to current project managers and their associated tech leads, as well as those that are in the project approval pipeline. This will at least generate some just-in-time consumers. While this may not yield the best service, it’s a start. Once some higher level analysts efforts have taken place to segment the business into business domains, then the right marketing targets may be more clearly understood.

I actually had to think harder than normal for a title for this entry because as I suspected, I had a previous post that had the title of “Importance of Identity.” That post merely talked about the need to get identity on your service messages and some of the challenges associated with defining what that identity should be. This post, however, discusses identity in a different light.

It occurred to me recently that we’re on a path where having an accurate representation of the organization will be absolutely critical to IT success. Organizations that can’t keep ActiveDirectory or their favorite LDAP up to date with the organizational changes that are always occurring with find themselves saddled with a boat anchor. Organizations that are able to keep their identity stores accurate and up to date will find themselves with a significant advantage. An accurate identity store is critical to the successful adoption of BPM technology. While that may be more emerging, think about your operations staff and the need for accurate roles associated with the support of your applications and infrastructure. One reorg of operations and the whole thing could fall apart with escalation paths no longer in existence, incorrect reporting paths, and more.

So, before you go gung-ho with BPM adoption, take a good look at your identity stores and make sure that you’ve got good processes in place to keep it up to date. Perhaps that should be the first place you look to leverage the BPM technology itself!

While I don’t normally comment on press releases that I occasionally receive in email, one tidbit in a release from Active Endpoints about ActiveVOS™ 5.0 caught my eye:

Active Endpoints, Inc. (www.activevos.com), inventor of visual orchestration systems (VOS), today announced the general availability of ActiveVOS™ 5.0. …

Scenario testing and remote debugging. ActiveVOS 5.0 fundamentally and completely solves a major pain experienced by all developers: the question of how to adequately test loosely-coupled, message-based applications. ActiveVOS 5.0 includes a new BUnit (or “BPEL unit test”) function, which allows developers to simulate the entire orchestration offline, including the ability to insert sample data into the application. A BUnit can be created by simply recording a simulation in the ActiveVOS 5.0 Designer. Multiple BUnits can be combined into BSuites, or collections of smaller simulations, to build up entire test suites. Once deployed into a production environment, ActiveVOS 5.0 delivers precisely the same experience for testing and debugging a production orchestration as it does for an application in development. Remote debugging includes the ability to inspect and/or alter message input and output, dynamically change endpoint references and alter people assignments in the application.

Back in November, in my post titled Test Driven Model Development, I lamented the fact that when a new development paradigm comes along, like the graphical environments common in BPM tooling, we run the risk of taking one or more steps backward in our SDLC technologies. I used the example of test-driven development as an example. As a result, I’m very happy to see a vendor in this space emphasizing this capability in their product. While it may not make a big difference in the business solutions out there, things like this can go a long way in getting some of the hard-core Java programmers to actually give some of these model-driven tools a shot.

Here are the SOA, BPM, and EA events coming up in March. If you want your events to be included, please send me the information at soaevents at biske dot com. I also try to include events that I receive in my normal email accounts as a result of all of the marketing lists I’m already on. For the most up to date list as well as the details and registration links, please consult my events page. This is just the beginning of the month summary that I post to keep it fresh in people’s minds.

• 3/3: ZapThink Practical SOA: Pharmaceutical and Health Care
• 3/4: Webinar: Implementing Information as a Service
• 3/6: Global 360/Corticon Seminar: Best Practices for Optimizing Business Processes
• 3/10 – 3/13: OMG / SOA Consortium Technical Meeting, Washington DC
• 3/10: Webinar: Telelogic Best Practices in EA and Business Process Analysis
• 3/11: BPM Round Table, Washington DC
• 3/12 – 3/14: ZapThink LZA Bootcamp, Sydney, Australia
• 3/13: Webinar: Information Integrity in SOA
• 3/16 – 3/20: DAMA International Symposium, San Diego, CA
• 3/18: ZapThink Practical SOA, Australia
• 3/18: Webinar: BDM with BPM and SOA
• 3/19: Webinar: Pega, 5 Principles for Success with Model-Driven Development
• 3/19: Webinar: AIIM Webinar: Records Retention
• 3/19: Webinar: What is Business Architecture and Why Has It Become So Important?
• 3/19: Webinar: Live Roundtable: SOA and Web 2.0
• 3/20: Webinar: Best Practices for Building BPM and SOA Centers of Excellence
• 3/24: Webinar: Telelogic Best Practices in EA and Business Process Analysis
• 3/25: ZapThink Practical SOA: Governance, Quality, and Management, New York, NY
• 3/26: Webinar: AIIM Webinar: Proactive eDiscovery
• 3/31 – 4/2: BPM Iberia – Lisbon

I attended a presentation from Raul Camacho from the SOA Solutions group of Microsoft Consulting Services yesterday. The talk provided some good details on some of the technical challenges associated with lifecycle events associated with services, such as the inevitable changes to the interface, adding new consumers, etc., but I actually thought it was pretty weak on the topic. I thought for sure that I had a blog post on the subject, but I was surprised to find that I didn’t. Some time ago (January of 2007, to be specific) I indicated that I would have a dedicated post, but the only thing I found was my post on product centered development versus project-centered development. While that post has many of the same elements, I thought I’d put together a more focused post.

The first point, and a very important point, is that the service lifecycle is not a project lifecycle or just the SDLC for a service. It is a continuous process that begins when a service is identified and ends when the last version of a service is decommissioned from production. In between, you’ll have many SDLC efforts associated with each version of the service. I presented this in my previous post like this.

Raul’s talk gave a very similar view, again presenting the lifecycle as a circle with an on-ramp of service identification. The activities he mentioned in his talk were:

• Service Identification
• Service Development
• Service Provisioning
• Service Consumption
• Service Management

There’s many similarities between these two. We both had the same on-ramp. I broke out the SDLC into a bit more detail with steps of release definition, development, testing, and deployment, while Raul bundled these into two steps of development and provisioning. We both had management steps, although mine labeled it as the triple-M of monitoring, marketing, and management. The one key difference was Raul’s inclusion of a step of service consumption. I didn’t include this in my lifecycle, since, in my opinion, service consumption is associated with the lifecycle of the consumer, not with the lifecycle of the service. That being said, service consumption can not be ignored, and is often the event that will trigger a new release of the service, so I can understand why one may want to include it in the picture.

The thing that Raul did not mention in his talk, which I feel is an important aspect, is the role of the Service Manager. We don’t need Service Lifecycle Management if there isn’t a Service Manager to manage it. Things fall apart when there isn’t clear lines of responsibility for all things about a service. For example, a frequent occurrence is where the first version of a service is built by a team that is responsible for both the initial consumer and the service. If service ownership and management falls to this team, it is at best a secondary interest, because their real focus was on the delivery of the service consumer, not of the service itself. A second approach that many organizations may take is to create a centralized service team. While this ensures that the ownership question will get resolved, it has scalability problems because centralized teams are usually created based upon technical expertise. Service ownership and management has more to do with the business capability being provided by the service than it does with the underlying technologies used to implement it. So, once again, there are risks that a centralized group will lack the business domain knowledge required for effective management. My recommendation is to align service management along functional domains. A service manager will likely manage multiple services, simply because there will be services that change frequently and some that change infrequently, so having one manager per service won’t make for an even distribution of work. The one thing to avoid, however, is to have the same person managing both the service and the consumer of a service. Even where there is very high confidence that a service will only have one consumer for the foreseeable future, I think it’s preferable to set the standard of separation of consumer from provider.

So, beyond managing the releases of each service, what does a service manager do? Well, again, it comes back to the three M’s: Monitor, Market, and Manage. Monitoring is about keeping an eye on how the service is behaving and being used by consumers. It has to be a daily practice, not one that only happens when a red light goes off indicating a problem. Marketing is about seeking out new consumers of the service. Whether it’s for internal or external use, marketing is a critical factor in achieving reuse of the service. If no one knows about it, it’s unlikely to be reused. Finally, the manage component is all about the consumer-provider relationship. It will involve bringing on new consumers found through marketing and communication which may result in a new version, provisioning of additional capacity, or minimally the configuration of the infrastructure to enforce the policies of the service contract. It will also involve discussions with existing consumers based on the trends observed through the monitoring activity. What’s bad is when there is no management. I always like to ask people to think about using an external service provider. Even in cases where the service is a commodity with very predictable performance, such as getting electricity into your house, there still needs to be some communication to keep the relationship healthy.

I hope this gives you an idea of my take on service lifecycle management. If there’s more that you’d like to hear, don’t hesitate to leave me a comment or send me an email.

Nick Malik posted a great followup comment to my last post on service contracts. For all of you who just follow my blog via the RSS feed, I thought I’d repost the comment here.

The fascinating thing about service contract standardization, a point that you hit on at the end of your post, is that it is not substantially different from the standardization of terms and conditions that occurs for legal agreements or sales agreements in an organization.

I am a SOA architect and a member of my Enterprise Architecture team, as you are, but I’m also intimately familiar with solutions that perform Contract Generation from Templates in the Legal and Sales agreements for a company. My employer sells over 80% of their products through the use of signed agreements. When you run $3B of revenue, per month, through agreements, standardization is not just useful. It is essential.

When you sign an agreement, you may sign more than one. They are called “multi-tier� agreements, in that an agreement requires that a prior one is signed, in a chain. There are also “associated agreements� that are brought together to form an “agreement package�. When you last bought a car, and you walked out with 10 different signed documents, you experienced the agreement package firsthand.

These two concepts can be leveraged for SOA governance in terms of agreements existing in a multi-tier environment, as well as services existing in an ecosystem of agreements that are part of an associated package.

For example, you could have one of four different supporting agreements that the deployment team must agree to as part of the package. All four could rely on the same “common terms and taxonomy� agreement that every development and deployment team signs (authored by Enterprise Architecture, of course). And you could have a pair of agreements that influence the service itself: one agreement that all consumers must sign that governs the behavioural aspects of the service for all consumers, and another agreement that can be customized that governs the information, load, and SLA issues for each provider-consumer pair.

If this kind of work is built using an automated agreement management system, then the metadata for an agreement package can easily be extracted and consumed by automated governance monitoring systems. We certainly feed our internal ERP system with metadata from our sales agreements.

Something to think about…

In an email exchange with David Linthicum and Jason Bloomberg of ZapThink in response to Dave’s last podcast (big thanks to Dave for the shout-out and the nice comments about me in the episode), I made some references to the role of the service contract and decided that it was a great topic for a blog entry.

In the context of SOA Governance, my opinion is that the service contract is the “container” of policy that governs behavior at both design-time and run-time. According to Merriam-Webster, a contract is “a binding agreement between two or more persons or parties; especially : one legally enforceable.” Another definition from Merriam-Webster is “an order or arrangement for a hired assassin to kill someone” which could certainly have implications on SOA efforts, but I’m going to use the first definition. The key part of the definition is “two or more persons or parties.” In the SOA world, this means that in order to have a service contract, I need both a service consumer and a service provider. Unfortunately, the conversations around “contract-first development” that were dominant in the early days caused people to focus on one party, the service provider, when discussing contracts. If we get back to the notion of a contract as a binding agreement between two parties, and going a step further by saying that the agreement is specified through policies, the relationship between the service contract and design and run time governance should become much clearer.

First, while I picked on “contract-first development” earlier, the functional interface is absolutely part of the contract. Rather than be an agreement between designers and developers, however, it’s an agreement on between a consumer and a provider on the structure of the messages. If I am a service provider and I have two consumers of the service, it’s entirely possible that I expose slightly different functional interfaces to those consumers. I may choose to hide certain operations or pieces of information from one consumer (which may certainly be the case where one consumer is internal and another consumer is external). These may have an impact at design-time, because there is a handoff from the functional interface policies in the service contract to the specifications given to a development team or an integration team. Beyond this, however, there are non-functional policies that must be in the contract. How will the service be secured? What’s the load that the consumer will place on the service? What’s the expected response time from the provider? What are the notification policies in the event of a service failure? What are the implications when a consumer exceeds its expected load? Clearly, many of these policies will be enforced through run-time infrastructure. Some policies aren’t enforced on each request, but have implications on what goes on in a request, such as usage reporting policies. My service contract should state what reports will be provided to a particular consumer. This now implies that the run-time infrastructure must be able to collect metrics on service usage, by consumer. Those policies may ripple into a business process that orchestrates the automated construction and distribution of those usage reports. Hopefully, it’s also clear that a service contract exists between a single consumer and a single provider. While each party may bring a template to the table, much as a lawyer may have a template for a legal document like a will, the specific policies will vary by consumer. One consumer may only send 10,000 requests a day, another consumer may send 10,000 requests an hour. Policies around expected load may then be enforced by your routing infrastructure for traffic prioritization, so that any significant deviation from these expected load don’t starve out the other consumers.

The last comment I’d like to make is that there are definitely policies that exist outside of the service contract that influence design-time and run-time, so don’t think that the service contract is the container of all policies. I ran into this while I was consulting when I was thinking that the service contract could be used as a handoff document between the development team and the deployment team in Operations. What became evident was that policies that govern service deployment in the enterprise were independent of any particular consumer. So, while an ESB or XML appliance may enforce the service contract policies around security, they also take care of load balancing requests across the multiple service endpoints that may exist. Since those endpoints process requests for any consumer, the policies that tell a deployment team how to configure the load balancing infrastructure aren’t tied to any particular service contract. This had now become a situation where the service contract was trying to do too much. In addition to being the policies that govern the consumer-provider relationship, it was also trying to be the container for turnover instructions between development and deployment, and a single document couldn’t do both well.

Where I think we need to get to is where we’ve got some abstractions between these things. We need to separate policy management (the definition and storage of policies) from policy enforcement/utilization. Policy enforcement requires that I group policies for a specific purpose, and some of those policies may be applicable in multiple domains. Getting to this separation of management from enforcement, however, will likely require standardization in how we define policies, and they simply don’t exist. Policies wind up being tightly coupled to the enforcement points, making it difficult to consume them for other purposes. Of course, the organizational culture needed to support this mentality is far behind the technology capabilities, so these efforts will be slow in coming, but as the dependencies increase in our solutions over time, we’ll see more and more progress in this space. To sum it up, my short term guidance is to always think of the service contract in terms of a single consumer and a single provider, and as a collection of policies that govern the interaction. If you start with that approach, you’ll be well positioned as we move forward.

Recent articles/blogs from Michael Meehan and Dan Foody both emphasized the typically negative connotations associated with governance. Michael compared governance to getting a colonoscopy with an IMAX camera, while Dan pointed out that “governance is about defining the box that you’re not allowed to think outside.”

I won’t argue that most people have a very negative reaction to the term, but there’s a key point that missing in the discussion. Governance is about the people, policies, and processes put in place to obtain desired behaviors. Governance does not have to be about command and control. If your desired behavior is complete freedom to try anything and everything, clearly, you don’t want rigid command and control structures. If your goal is to be a very innovative company, but your command and control structures prevent you from doing so in a timely fashion, that’s not a problem with governance per se, but with the governance model you’ve chosen. While the United States has clear separation between church and state, the same can’t be said for many other countries in the world. If the desired behavior for those countries is strict adherence to religious principles, then clearly there will be some challenges in applying the United States’ governance model to those countries. It doesn’t mean those countries don’t need governance, it means they need a different style of governance.

One of my earliest posts on the subject of governance emphasized that the most important thing is that your governance model match your corporate culture. If it doesn’t it’s not going to work. I’ll also add to it that your employees need to accept that corporate culture as well. If the people in the company don’t agree with the desired behaviors that the leaders have established, you’re going to have problems. We need to stop attacking governance, and instead educate our staff on the desired behaviors and why they’re important so that people will want to comply. It must be the path of least resistance. That’s still governance, though. It’s just governance done right.

A combination of things, including the Workday acquisition of CapeClear and doing my taxes, got me thinking a bit more deeply about integration as a service (IaaS). When I have my large enterprise hat on, I’ll admit that IaaS doesn’t excite me very much. The main reason for this starts with what I envision when I see this, however. What I picture is taking all of the mapping exercises, transformations, etc. that are done today using some visual tool from an EAI vendor and moving it to a web-based system. I still need to do all of the work to wire things up, but the actual processing behind all of this goes on outside the firewall. This certainly doesn’t make sense if all of my integration points exist within the firewall, and I’d even go further to say that it’s not very attractive when only one of the integration points is outside of the firewall. Why? Well, there are already providers out there that will not only handle the processing, but also do the mapping. Essentially, I give them information on how I want my data, and they take care of doing all the dirty work to map to the other integration point. So, from this perspective, my definition of IaaS actually winds up having me do more work than the traditional “integration as a service” providers.

If all of my integration points are outside of the firewall, things get a bit more interesting. This occurred to me when I was doing my taxes. Most of the leading tax software providers have both desktop versions and online versions. In the case of the online version, the data resides in the cloud. The data behind the tax return must integrate with data from other cloud providers, such as payroll systems and financial services companies. While this integration works well, the same situation isn’t as good for general personal financial planning software. Take Quicken for example. In the desktop version, some of my financial data can’t be automatically imported into Quicken, because the financial provider hasn’t exposed the information according to the necessary standards. Interestingly, however, when I looked into the recently announced Quicken Online, some of the financial providers that didn’t work in the desktop version were able to be integrated into the online version. I suspect that there may be some screen scraping going on, as is the case for many financial aggregators like Yahoo Finance.

Anyway, the short point to that long winded paragraph is that when all of the data exists in the cloud, there’s no doubt that the need to integrate that data for one purpose or another will soon follow. The key question then, however, is whether services should be provided to allow individuals to create their own integration paths, or if the service providers will be expected to simply integrate with other leading offerings, much in the same way we expect the software we install inside our firewalls or inside our homes to integrate. I suspect it will be the latter. Whether its the consumer market or the enterprise market, I think we all want the difficult integration problems to be handled for us. The vendors that are able to do so, will be successful.

I saw the announcements yesterday that Workday had acquired CapeClear. This is an interesting acquisition. At first glance, I thought this was just normal evolution in the ERP space, except that Workday just happens to be a hosted solution. After all, Oracle and SAP both have integration products that enable integration with their backend applications, so why not Workday? The part of this that is a bit distressing, however, is that Workday is a new entrant into this space. A big reason that these integration products exist is because every application had its own proprietary integration approach. As the need to integrate became more and more important, the desire to standardize the integration approaches increased, and led to technologies like Web Services. Workday, as a relatively new player in the space, should have been able to go with standardized integration approaches from the start, significantly reducing the need for integration middleware. There will always be a need for something in the middle, but it should start to look more like network devices than complex middleware. If this theory is true, then why would Workday need to acquire an integration middleware company like CapeClear? Perhaps the only reasoning in this is that CapeClear had always tried to be more that just mediation fabric. After all, I had railed on CapeClear in the past when David Clarke said, “We consider ESB the principal container for business logic. This is the next generation application server.” It’s likely that Workday actually used them in this fashion, rather than as a lightweight mediation fabric that I prefer. If that’s the case, then it’s entirely possible that Workday was in a position where it would be too expensive to migrate to a traditional Java application server, and CapeClear was struggling being one of the few SOA startups left with rough economic times ahead. The best course for both parties, therefore, is the actions that have taken place. It will remain to be seen how well Workday can support the existing CapeClear customers who are using it as a mediation fabric, since their bread and butter is the ERP system, not middleware.

I was surprised at David Linthicum’s latest blog entry. Normally, he’s pretty good about emphasizing that you can’t buy an SOA, but in his “Defining SOA Governance” post, a lot of the conversation was very tool-centric. He stated the following.

Key components of design time SOA governance include:

• A registry and/or repository for the tracking of service design, management, policy, security, and testing artifacts.
• Design tools, including service modeling, dependency tracking, policy creation and management, and other tools that assist in the design of services.
• Deployment tools, including service deployment, typically through binding with external development environments.
• Links to testing tools and services, providing the developer/designer the ability to create a test plan and testing scenarios, and then leverage service testing technology.

On the runtime governance side of things, he did preface the tool capability discussion with this statement: “Thus, runtime governance is the process of enforcing and implementing those policies at service run time.”

I’ve said it before, and I’ll say it again. Governance is about people, policies, and process. Tooling really only comes into play when you start looking at the process portion of the equation. I don’t want to dismiss tooling, because it absolutely is an important part of the governance equation, but if you don’t have the people or the policies, tools won’t help.

The other thing that I want to call out is how “SOA Management” has morphed into “Runtime SOA Governance.” Back in March of 2006, I was on a panel at the InfoWorld SOA Executive Forum and teased the panelist from AmberPoint about hijacking the term, but I’ve relented a bit simply because these terms all have preconceived notions about them, and the use of the term governance probably gets people thinking in the right direction. For many people, SOA Management may imply the kind of passive monitoring that people associated with traditional systems management (not that it should be passive, but that’s the perception many have). Runtime SOA Governance, however, is rooted in the notion of the active enforcement of policies associated with the consumer-provider interaction. If a change in marketing terms helps people understand it, I’m all for it.

But back to the main subject… whether it’s runtime or design time, there’s still a need to understand the policies and people/parties involved. If you don’t understand the concepts associated with a service contract (and it’s not just the functional interface) and have people on the both sides of the interaction who care about them, governance tools aren’t going to do you any good. If you don’t have people defining the enterprise policies associated with choosing and defining services, again, the tools aren’t going to be used effectively.

Here are the SOA, BPM, and EA events coming up in February. If you want your events to be included, please send me the information at soaevents at biske dot com. I also try to include events that I receive in my normal email accounts as a result of all of the marketing lists I’m already on. For the most up to date list as well as the details and registration links, please consult my events page. This is just the beginning of the month summary that I post to keep it fresh in people’s minds.

• 2/4 – 2/6: Gartner BPM Summit
• 2/5: ZapThink Practical SOA: Energy and Utilities
• 2/7 – 2/8: Forrester’s Enterprise Architecture Forum 2008
• 2/11: Web Services on Wall Street
• 2/13 – 2/15: ARIS ProcessWorld
• 2/13: ZapThink Webinar: Leverage Document Centric SOA for Competitive Advantage
• 2/19: Webinar: Integrated SOA Governance
• 2/25 – 2/28: BPTG’s Business Process Transformation – BPM Practitioner Course
• 2/25 – 2/27: Global Excellence Awards in BPM & BPM Technology Showcase
• 2/26 – 2/29: ZapThink LZA Bootcamp

A recent conversation brought up another real world governance example that I think is an excellent illustration of where we need to get to in our SOA (and IT) governance efforts. The town I live in borders the Mississippi River. In early 2007, the city council signed a master development agreement with an area developer to develop the “bottoms” area, essentially farm land that butts up against the river levees. Shortly after that, elections changed the makeup of that council, and now there is a majority that is against this development and has been doing everything they can to keep things from moving forward, so much so, that the developer has a suit against the city. That’s not integral to this post, however. What has recently been the topic of discussion is the levee system. The Army Corps
of Engineers is going to decertify the levees and require the local municipalities to pay for upgrades. So, the question at hand is, “Who’s going to pay for this?” The master development agreement calls for the developer to pay for the improvements as there’s a vested interest there, since the levees would be protecting the businesses in the development. If that agreement falls apart with the current issues, some alternative method of funding would need to be found. The city and/or county really only have taxes at their disposal. Either sales taxes or property taxes would need to be increased.

So what does this have to do with SOA and IT Governance? Well, look at it this way. The master development agreement is like an IT project in many companies these days. The primary focus is on delivering some new businesses to the area, but as part of that, there’s some infrastructure work that needs to happen. The levee upgrade is a clear infrastructure need, but it’s extremely difficult to find funding for it without bundling it in with some other more tangible effort that is closer to the end user, in this case the residents of the surrounding area.

Now, if this effort were the typical IT project, What would happen? The developer would build their shopping centers, distribution centers, warehouses, whatever, but they would put in only the infrastructure required for that. Imagine seeing a levee that surrounds their development, but as soon as you leave their property, we go back to some crumbling infrastructure which may actually now be worse, because the added strength of the new section puts more pressure on the aging infrastructure. Now, this won’t actually happen, because the Army Corps of Engineers have regulations that will ensure that the levee upgrades are built properly.

What we need to happen in the IT world is to have the same types of regulations so that when projects go and request funding, the scope is such that funding includes building services and infrastructure in the right way, rather than how it is today where the scope is set solely to the immediate needs of the project. The problem is that we don’t have the regulations that can actually set the scope. We instead have regulations that begin with something like, “If you build a shared service…” which immediately gives the project managers or proposers a way out, because they’ll claim that their services aren’t going to be shared, when in fact, they probably simply haven’t looked into whether there is a shared need, simply because of the risk of expanded scope. I think some form of business domain model as I’ve discussed in the past (here and here) that classifies domains of capabilities from the perspective of “share-ability” is a step in the right direction.

Thanks to Gartner and the SOA Consortium, the panel discussion I was part of in December at the Gartner Enterprise Architecture Summit is now available as a podcast. I’ve referenced it as an enclosure in this entry, so if you subscribe to my normal blog feed in iTunes, you should get it. If you have difficulty, you can access the MP3 file directly here. For all the details on the session, I encourage you to read Dr. Richard Soley’s post over at the SOA Consortium blog.

While I always take vendor postings with a grain of salt, David Bressler of Progress Software had a very good post entitled, “We’re Running Out of Words II.” Cut through the vendor-specific stuff, and the message is that routing all of your requests through a centralized process management hub in order to gain visibility may not be a good thing. In his example, he tells the story of a company that took some existing processes and, in order to gain visibility into the execution of the process steps, externalized the process orchestration into a BPM tool. In doing so, the performance of the overall process took a big hit.

To me, scenarios like this are indicative of one major problem: we don’t think about management capabilities. Solution development is overwhelmingly focused on getting functional capabilities out the door. Obviously, it should be, but more often than not there is no instrumentation. How can we possibly claim that a solution delivers business value over time if there is no instrumentation to provide metrics? Independent of whether external process management is involved, gateway-based interception is used versus agent-based approaches, etc. we need to begin with the generation of metrics. If you’re not generating metrics, you’re going to have poor, or even worse, no visibility.

Unfortunately, all too often, we’re only focused on problem management and resolution, and the absence of metrics only comes to light if something goes wrong and we need to diagnose the situation. To this, I come back to my earlier statement. How can we have any confidence in saying that things are working properly and providing value without metrics?

Interestingly, once you have metrics, the relationships (and potentially collisions) between the worlds of enterprise system management, business process management, web service management, business activity monitoring, and business intelligence start to come together, as I’ve discussed in the past. I’ve seen one company take metrics from a BPM tool and placing them into their data warehouse for analysis and reporting from their business intelligence system. Ultimately, I’d love to see a differentiation based upon what you do with the data, rather than on the mere ability to collect the data.