Todd Biske: Outside the Box

Archive for the ‘Governance’ Category

The Power Breakfast is over and done. If you saw me there, hopefully you got something out of the conversation, although it was unfortunate that the session wound up being four independent case studies without a lot of time for audience questions, rather than a discussion among the panelists and the audience. Despite this fact, there were some good points that came out, and I wanted to highlight the ones that struck home with me here:

• First, as the preliminary results of the SOA Consortium funding survey have indicated, there’s no uniform initial approach to SOA. Some leverage an SOA program, some expect service development out of existing projects, etc. One of the panelists, Leo Schuster from National Bank, indicated that they had an SOA Program. In my experience, I’ve never encountered a funded program whose sole goal was to produce SOA. I’ve always worked with business projects/programs that wanted to leverage services as part of their efforts. Personally, I prefer the latter, because as Victor Harrison from CSC pointed out, programs end. Will your SOA efforts ever stop? They shouldn’t. So, while you may have solved the initial funding problem, you haven’t solved the ongoing funding problem.
• Second, metrics are critical. Every single one of us mentioned metrics. Without them, how can we say that we’ve accomplished anything? The oft-mentioned notion of agility is frequently associated with the time to deliver a particular solution. To show that we’re more agile, we need metrics of how things have been and how things are in the future. Beyond efficiency metrics, there’s also metrics associated with service consumption: number of consumers, usage by consumers, min/avg/max response time, etc. In my own experience, merely making these metrics available, and now at a finer level of granularity than the entry points into a web application were very beneficial. This visibility didn’t exist before, so anything was better than nothing. If you’re already collecting this, now you can start to do baseline comparisons to show improvement.
• Third, service ownership is key. It was great to hear someone else say that the project-based culture of IT can be in impediment to SOA success. While your SOA shouldn’t have a lifecycle, individual services do have a lifecycle. If you simply have someone who is responsible for service lifecycle management, you’ve got the key piece of the puzzle for many of the discussions around service granularity, funding, etc. If you don’t have a service owner independent of the consumers, then each consumer is going to try to push things in the direction that has the most benefit for their project, but yet no one will own the service. This in-fighting can be very detrimental to SOA.
• Finally, lets get Service Portfolio Management (SPM) right from the get-go. There are many sessions here at the summit concerned about application portfolio management. We’re all behind the eight-ball here as all of these applications have been built/bought over 5, 10, 15, 25, … years without anyone managing them as a whole. The last thing we want to do is repeat the process all over again with services. We’re at the early stages of SOA and can now do it right. Anyone saying that they don’t need a registry/repository yet because they haven’t reached “critical mass” is potentially making a big mistake. They only need to look at their struggles in trying to do application portfolio management as an example of the path they are heading down.

To those who got up early for breakfast and attended, I hope you found some nuggets in the presentation that were valuable from me or from the other speakers. I’d be happy to have followup conversations with anyone who felt like their questions are still unanswered, or those who want more depth on some of the things they heard.

In this session, Frank Kenney and Val Sribar, are discussing agility and application development in the context of a fictitious fable of a web unit in a business organization that is operating in an agile manner and how it works with the traditional application development organization. While I find the fable a bit far fetched, since I’ve yet to see a business unit pushing agile processes on IT, it’s usually the opposite, there is one key point that they made. They called out that not all business units may be ready for an agile approach, simply because they don’t change that rapidly. Hopefully, this will be a common theme throughout the conference. Any approach that claims to be one size fits all is probably going to run into problems. Whether it is agile, REST, Web Services, SOA, EDA, etc., there are always scenarios where it works well, and scenarios where it doesn’t work so well. A key to success is knowing not only how to apply a particular technique, but knowing when to apply it.

Further in on the presentation, they went through some different areas and discussed how they contribute to agility. In the subject of business process actions, they called out that there should be a single business person clearly in charge of a particular process. I think this applies not only for processes, but for services as well. There needs to be a service manager for each service in the organization. An individual may manage multiple services, but without a service manager, there will be problems.

On the subject of reuse, Frank Kenney pointed out that incentives, metrics, testing, methodology, and inventory management (registry/repository) is necessary to make it a reality. I think this is a good list, although I would have liked to see marketing included. While a registry/repository and methodology do focus on the consumption side, if a service or reusable asset is placed out there in a way that makes it difficult to find, the burden is on the provider to improve it, which involves marketing.

Near the end of the presentation, Val and Frank presented an “Application Group of the Future.” Val called out breaking out groups for User Experience, Business Processes, Business Information, an Integration Competency Center, Business Relationships, Software Quality, and Information Infrastructure. These groups would all be operating in an agile manner. In addition, the traditional development groups would also exist. These groups would all leverage common environments to support their efforts. There was one thing about this picture that I disagreed with, which was the placement of SOA. They sprinkled the entire organization with SOA experience, which makes sense, but they didn’t call out the separation between organizations focused on service consumption and organizations focused on service production. If anything, service production (unless it happened to be an automated business process) was buried in the tradition development group, and I don’t think that will cut it. On the bright side, one of the things that they called out which was very interesting, was the need for dynamic budgeting. Val made a comment that the web and mobile applications are challenging the way we book business early in the presentation. This need for dynamic budgeting is an analogous example to how a new agile IT, challenges the way we traditionally do budgeting. Annual budgets won’t cut it, we’ll need to make it more dynamic and responsive in real-time.

In his links post, James McGovern had some comments on my Long Tail of Applications post. He stated:

Todd Biske wants to eliminate the term “application” as it implies a monolith. I would like to point out to Todd that there is another usage of the word that still remains important which primarily indicates a funding model. It is possible and viable to build a great SOA while still letting the finance folks think in terms of applications. Removing the term from architects is a great thing but very disruptive for other parts of the enterprise.

I’m glad James brought up the whole funding model aspect of it. As he rightly calls out, the removal of the term “application” is probably not terribly disruptive for architects, it’s very disruptive for the rest of the organization. How many IT departments have organizations with application development in the title? How many project charters contain the words “…deliver an application…” in their text? If you agree with the premise that, in general, IT is not delivering at the level it could be, then something needs to change. We need a disruption. If that disruption ripples all the way up to the way projects are defined and funded, then so be it. This type of change doesn’t happen overnight, and is not going to be without many bumps in the road.

Does everyone need this type of disruption? Perhaps not, but I think that it’s a very difficult thing to determine. If you’ve read The Innovator’s Dilemma or The Innovator’s Solution, you know what I’m talking about. While I can envision what an organization that has truly embraced SOA and BPM technologies might look like, it’s far more challenging to determine what the path to that state is, which can also leave you wondering whether it’s even necessary. Ultimately, it only becomes necessary when some unknown competitor embraces it and starts beating the pants off of you. Finding that first example that justifies trying a new way of doing things is hard enough, finding the second and the third to justifying sustaining it is even tougher. I’m up for the challenge, though!

Jason Bloomberg of ZapThink, in their latest ZapFlash, put a new spin on their old concept of the SOA champion and put forth a job description for VP of SOA. While he certainly suggested a good base salary for the position, I question whether a permanent, executive position around SOA makes sense?

If you look at the job responsibilities listed, the question I ask is not whether these tasks are needed, but rather, whether a dedicated person is needed for all of them. Let’s look at a few of them:

Provide executive-level management leadership to all architecture efforts across the enterprise. The directors of Business Architecture, Enterprise Architecture, Technical Architecture, Data Architecture, and Network Architecture will all be your direct reports.
Don’t we already have this? It sounds like a Chief Architect to me.

Drive all Business Process Management (BPM) initiatives enterprisewide. Coordinate with process specialists across all lines of business, and drive architectural approaches to business process.
Again, to me, this sounds like the responsibility of the COO.

Establish and drive a SOA governance board that incorporates the existing IT governance board.
This is the only one that I simply disagree with. If we’re speaking in terms of IT governance as defined in Peter Weill’s book, I think the IT Governance Board should be factoring SOA strategy and governance into their decisions. That is, IT Governance subsumes SOA governance, not vice versa. Of course, there are also aspects of SOA governance that are implemented at a much lower level within projects to ensure consistency of service definitions, etc. Once again, however, this should be handled by the existing technology governance processes. We’re merely adding some additional criteria for them to be applying to projects.

Establish and lead the SOA Center of Excellence across the enterprise to pull together and establish core architectural best practices for the entire organization. Develop and enforce a mandate for conformance to such best practices.
This gets into the technical governance I just mentioned. I certainly agree that a SOA COE can be a good thing, but you certainly don’t need a new position just to manage in it. Why wouldn’t the Chief Architect or a delegate lead the COE as part of their day to day responsibilities?

Manage a budget that is not tied to individual line-of-business projects, but is rather earmarked for cross-departmental SOA/BPM initiatives that drive business value for the enterprise as a whole.
The question here is whether or not the current organizational structure prevents cross-departmental initiatives from being funded and managed properly. It implies that the individual LOBs will be more concerned about their own needs, and not that of the broader enterprise. If the corporate governance principles and goals have stated that the use of more shared, cross-cutting tehcnologies are needed, then it’s really a governance problem. While an organizational change can assist, you still need to ensure that LOB managers are making decisions in line with the corporate goals, rather than that of their individual LOBs.

Work closely with the VP of Project Management to insure close cooperation between architecture and project management teams, and to improve project management policies.
Once again, why can’t this be done by the existing architectural leadership?

There were additional items, but my thoughts kept coming back to “shouldn’t someone already be doing this?” If the answer to this is “no,” then you must ask yourself whether or not you’re really committed to SOA adoption. If you feel that you are, but don’t have anyone assigned to these responsibilities, does the creation of a new position make sense? For example, if the organization struggles with getting LOB managers to produce cross-departmental solutions, will throwing one more person into the mix fix the problem, or just add another point of view?

As with many of the ZapThink ZapFlashes, there’s always a bit of controvery but lots of goodness. In this case, the articulation of the responsibilites associated with managing SOA adoption are excellent. Do you need a new position to do it? As with any organizational decision, it depends. If you are committed to SOA, but can’t make it happen simply because all of the possible candidates are simply to busy to take on these new responsibilities, then it might make sense (although you’ll probably need to add staff elsewhere, too, if people really are that busy). If you’re trying to use the position to resolve competiting priorities where there isn’t universal agreement on what the right thing to do for the enterprise is, a new position may not resolve it.

Robert Annett had a good post asking the question, “Do you need to build it yourself?” He states:

I’ve worked in many organisations and one thing that never ceases to amaze me is how often teams build unnecessary tools and frameworks … As a software/systems architect it’s your job to deliver value to the business you work for. Whenever someone suggests creating a bespoke tool ask yourself the following:

• Can I buy something to do this?
• Can I use or tailor an open source product?
• Can I use or create a template for a standard tool like Eclipse, Word or Excel?

I think that this is a challenge for corporate IT departments. I managed a frameworks team at a previous employer, and it’s extremely difficult to figure out where to invest your time. It was a continual game of guessing as to what things the vendors would add in the next release, what things would take off in open source community, and what things would be woefully underserved by third party products, whether open source/closed source or freeware/commercial products. It’s also not limited to low level frameworks, either, although the higher you go up toward business applications, the fewer choices there are and the more they cost.

Every company is going to be different with how they approach this problem, and a lot of it is dependent on company size. If there are IT resources to burn, you may be able to afford having a frameworks team, potentially even allowing developers to contribute to open source frameworks on company time. If IT resources are scarce, however, you have to balance the cost of a third party product and the cost of not having resources working on business solutions where custom coding is required.

I expect that to many of my readers, this is nothing new, but I do think that it a reminder now and then is good. Our responsibility is to deliver the best business solutions for appropriate costs, whether it is something off the shelf or something custom built. It is our job to be good corporate stewards and do our jobs wisely.

At the upcoming Gartner Application Architecture, Development and Integration Summit, I’ll be part of a panel discussion on funding SOA. I’ve previously posted on this in this entry, but I thought I’d bring it up again with the upcoming conference.

I’m very interested in hearing the experience of others on this topic. While there’s a lot of discussion about funding models, I still have yet to run into an organization that has had to actually implement one of these models. More often than not, I’ve seen one of two things:

1. A program of large enough scale where a number of services will be created with some in use by more than one consumer
2. Project-level SOA where a single project develops both the consumer and the service

There’s nothing wrong with either of these, but the thing to note is that these efforts did not require any change to the way funding of IT efforts occurs.

In discussing this with some colleagues, it seems that changes in how projects are funded really only comes about when reuse gets involved. In many ways, I feel that it is no different than dealing with shared (reused) infrastructure except that it is a bit more difficult to figure out how to partition the responsibilities.

A key question in all of this is how many services will be reused? If only 5% or 10% of services are reused, it is hard to justify changes to the funding model. But is this a chicken versus the egg scenario? Perhaps it is the way IT projects are defined to begin with which hampers the organization’s ability to identify services with reuse potential.

The point of this is that we’re still in the very early stages of SOA adoption. My sample base is very small, so I’m very interested in whether what I’m seeing is just an artifact of my small sample base, or if SOA funding is still a topic ahead of its time. This topic came up in a recent SOA Consortium call, and in order to reach out to a broader sampling base, I assisted in the development of a survey on the topic. It’s relatively high level and shouldn’t take very long to complete. I, and the other members of the SOA Consortium would certainly appreciate the input. It is intended for corporate SOA practitioners. You can access it here. Thanks!

InfoWorld published a collection of end-user stories on SOA this week, and the discussion on Leapfrog Enterprises caught my attention. In the article, Galen Gruman states that:

Leapfrog had many of the same goals that typify a typical SOA initiative: greater reuse of code, faster development time, and easier integration. But the company did not want to approach SOA as simply a changing of the guard for development tools and integration platforms. Instead it wanted to free its developers from conforming to a platform’s idea of best practices so they could focus on the applications’ functionality and use a wide range of development technologies as best for each job.

[Eugene] Ciurana [director of systems infrastructure for Leapfrog] did not want to force developers to all use the same transport. “The transport doesn’t matter,” he says. He chose to use the open source Mule ESB as a messaging backbone, relying on it to deal with transport interfaces. That way, “developers could focus as little as possible on the implementation of services,” he explains. Instead, their focus is on the functionality they are trying to achieve. The result is that developers tend to use HTTP as their transport mechanism, but some use REST (Representational State Transfer) and SOAP — “whatever works best or they’re most comfortable in.”

This caught my attention because it appears to be contrary to what I’ve seen at other organizations. Typically, the organization wants to constrain the platform to ease the integration problem and get away from the notion of “any-to-any” integration hubs. This may just be my misinterpretation, but it does raise an interesting question. How many constraints should be put in place? Interestingly, I’ve yet to run into an organization that has had to drive adoption of XML-based integration via enterprise architecture. The developers have slowly migrated away from whatever distributed object technology they were using and toward XML. The bigger challenge has been whether or not the XML contained the right information for broad consumption, not on whether or not XML was used. That being said, many EA teams are focused on the latter constraint (when to use XML or not). Knowing that there’s only so much that can be governed, what are the critical factors that EA teams should be making sure we get right as compared to the broad spectrum of things that we could be governing? Is it worth the pain to create policies regarding SOAP/HTTP versus XML/HTTP? The approach draws parallels to the big government/small government discussions that go on between the Democrats and the Republicans. The right answer is very dependent on the culture within IT, as I’ve stated often in this blog. Personally, I’m not a big fan of the integrate any-to-any approach. That being said, I also recognize that not everything is going to adhere to the constraints. I think it’s important to differentiate between your connectivity (mediation) infrastructure and your service enablement activities. Connectivity is about connecting two parties that adhere to the constraints for communication that have been adopted by the organization. Unless there’s only one way to communicate, there will be a need for mediation, for example, HTTP to JMS bridging. It’s important that the set of technologies be small, however. Service enablement is about taking something that doesn’t adhere to the standards and exposing it in a way that it does. We should strive to reduce the amount of service enablement over time, but the need for connectivity and mediation will always be there.

Okay, no bears, sorry. I read post from my good friend Jeff Schneider regarding SAP’s Enterprise Service Repository (ESR). He states:

At the core of the SAP SOA story is the Enterprise Service Repository (ESR). It is actually a combination of both registry and repository. The registry is a UDDI 3.0 implementation and has been tested to integrate with other registries such as Systinet. But the bulk of the work is in their repository. Unlike other commercial repositories, the first thing to notice is that SAP’s is pre-populated (full, not empty). It contains gobs of information on global data types, schemas, wsdl’s and similar artifacts relating to the SAP modules.

This now brings registry/repository into the mix of infrastructure products that SAP customers must make decisions regarding adoption and placement. Do they leverage what SAP provides, or do they go with more neutral products from a pure infrastructure provider such as BEA, HP, SOA Software, or SoftwareAG/WebMethods? The interesting thing with this particular space is that it’s not as simple as picking one. Jeff points out that the SAP ESR comes pre-populated with “gobs of information” on assets from the SAP modules. Choose something else, and this metadata goes away.

I hope that this may bring some much needed attention to the metadata integration/federation space. It’s not just a need to integrate across these competing products, but also a need to integrate with other metadata systems such as configuration management databases and development lifecycle solutions (Maven, Rational, Subversion, etc.). I called this Master Metadata Management in a previous post.

Back when Gartner was pushing the concept of the ESB heavily, I remember an opening keynote from Roy Schulte (I think) at a Web Services Summit in late 2005. He was emphasizing that an organization would have many ESBs that would need to interoperate. At this point, I don’t think that need is as critical as the need for our metadata systems to interoperate. You have to expect that as vendors of more vertical/business solutions start to expose their capabilities as services, they are likely to come with their own registry/repository containing their metadata, especially since there’s no standard way to just include this with a distribution and easily import it into a standalone RR. It would be great to see some pressure from the end-user community to start making some of this happen.

Mike “The Mad Greek” Kavis had an interesting post on SOA Lessons Learned over at IT Toolbox. First off, a big thank you to Mike for sharing some experiences about an effort that didn’t go the way it was originally planned. We can learn as much from these as we can from success stories.

The part of the post that caught my attention began with this line:

As we started the second wave of projects, I mandated that all code should be delivered with test harnesses, the build process should be 100% automated, and testing automation should be part of the project deliverables.

Mike went on to discuss how automation and governance were quickly forgotten when the schedule began to slip. The surprising thing to me was not that these aspects were dropped when the schedule began to slip, but that the implementation of things like continuous integration and testing automation were tied to the SOA effort to begin with.

To me, this was indicative of something that I’ve seen at a number of places which is to use “SOA” as the umbrella to fix all things in need of improvement within the software development process. Continuous integration is a great thing and everyone should be practicing it. But if you organization hasn’t adopted it or created a standard, repeatable way of doing it, don’t target your pilot for another key initiative (like SOA) to try to make it happen. SOA adoption is not dependent on having a continuous integration system. If you have it, will it make SOA easier? Yes, probably, but more so because it’s making all development easier, regardless of whether you’re practicing SOA or not. Give it its own pilot where it can be successful in a very managed fashion, and roll it out to the rest of the enterprise.

Part of the problem is finding ways to adopt these improvements in the development process. Is the business going to care about continuous integration? You can argue that they should, but it’s really about internal IT processes. All too often, IT is left to take the congressional lawmaker approach and find some big project that will be sure to be funded and then push everything but the kitchen sink into it under the radar. This creates additional risk and often results in the project team biting off more than they can chew. It’s unfortunate that IT frequently has little ability to improve on its internal processes due to the project-centric nature of its work. Take another support organization like HR. While I’m sure some work in HR work is project-driven, a lot of the work is day-to-day operations. My suspicion is that organizations that are focused more on fixed-cost day-to-day work like this probably have more ability to take on internal improvement initiatives.

My point of all this is that an organization has to be careful on what they try to take on. There’s always opportunities for improvement, and the point should be quality, not quantity. Trying to take on everything is unlikely to lead to success on any of it. Taking on a smaller set of goals and ensuring that you do them well is a safer approach.

I’m a frequent listener of Biotech Nation with Dr. Moira Gunn, available through IT Conversations, and have begun learning more about the world of biotechnology. In doing so, I realized that there are some good analogies between the practices in agriculture and what we need for governance from IT. I’m not the first one to use an agriculture analogy, as Neil Macehiter and Neil Ward-Dutton described the IT technology landscape as a garden in their book, “The Technology Garden: Cultivating Sustainable IT-Business Alignment.”

When we grow something, it starts with the seed. The seed needs to germinate to grow into something, be it corn, wheat, soybean, etc. There’s plenty of things that can go wrong at this stage, so seeds are normally pre-treated with some form of protection, such as an pesticide. Once the seed germinates and the plant begins to grow, the farmer or gardener now needs to worry about weeds and insects. In the past, this involved heavy doses of chemicals, be it pesticides or herbicides. The problem with these is that these chemicals tend to work selectively on particular bugs or particular weeds. If you had a different bug or a different weed, you threw more chemicals on it. Not only is there risk that these chemicals stay on the food that the plants produce, but also that the runoff does harm to the environment. Today, there are non-specific herbicides that will kill just about anything that is green and leafy. The problem with this is that is would normally kill the crop. To that end, biotechnology has allowed crops to be grown that have a resistance to these non-specific herbicides. The net result is that less chemicals are necessary. Rather than continual spraying of selective herbicides, a single spray of a non-specific herbicide can be used.

So how does this relate to governance? The seed is the typical IT project. This is the thing that we’re trying to grow. Pick the wrong seed and you have problems. If you don’t pre-treat the seed, it may never germinate. The theme here is that projects need to be positioned for success from the beginning. There are activities that take place at the inception (or even before) of a project that can make or break your efforts. Given that most IT shops are still very tactical in their activities, SOA adoption is still predominantly a bottom-up effort. If you don’t properly scope your projects, as well as watch it carefully at the beginning so that any service development efforts are properly scoped, your chances of SOA success (or long-lasting IT success for that matter), will be less.

Step two in the growing process was the use of biotechnology. These plants have genetics that create a resistance toward the bad things that could come along. In the IT project sense, this is all about making the right thing the path of least resistance. If you’re successful with doing this through education, tooling, and mentoring, you won’t need to worry about review committees and rigid processes, because your seed will naturally do the right thing.

Finally, there is still a need for some general oversight. Weeds can still grow in your garden, and those weeds can consume vital resources that your plants need to live. In the IT sense, this isn’t about the specific decisions on the projects, it’s about the resources allocated to the project. Even if your architecture is sound, if your resources aren’t empowered to do the right thing, or if they’re pulled in many directions by multiple projects, you’re going to have problems.

So, while many people have frequently used the city planning analogy for architecture and governance, I’m beginning to like the gardening/farming analogy even more. Create an environment that positions things for success, make the “right thing” the path of least resistance for the project, and finally, keep the weeds out, allowing the resources allocated to focus on making the project successful.

It’s about 6:20 in the morning, and I’m presently on a 5 hour bus ride with a bunch of IT staff headed toward some facilities of my employer to learn more about their business. Before I get into the topic for this entry, I certainly want to give kudos to my employer for setting up this opportunity for IT to learn more about the business. With the long bus ride, I’m trying to get caught up on some podcasts. I’m presently listening to a discussion on SOA Management with Jason Bloomberg of ZapThink and Dana Gardner of Interarbor Solutions.

In his intro, Dana Gardner used the oft-mentioned and maligned term, silo. For whatever reason, it suddenly occurred to me that we always work within silos. Organizational structures create silos. Projects create silos. Physical locations can create silos. So, perhaps the right discussion shouldn’t be how to eliminate silos, but rather, how to choose silos correctly, work appropriately within them, and know when to redefine them. I’ve commented a bit on how to organize things, specifically in my post on horizontal and vertical thinking. For this entry, I’d like to discuss the appropriate way to operate within a horizontal silo.

A horizontal silo is one where the services being provided from that silo have broad applicability. A very easy example most organizations should be familiar with is servers. While there are multiple products involved based on the type of processing (e.g. big number crunching versus simple transactional web forms), you’d be hard pressed to justify having every project select its own servers. This area isn’t without change, one only needs to look at the space of VMWare and its competitors to see this.

In many IT organizations, when a centralized group is established, the focus can often be on cost containment or reduction. For many horizontal domains, this makes a lot of sense, but there’s a risk associated with this. When a group focuses on cost reduction, this is frequently done at the expense of the customer. Cost reduction typically means more standardization and less customer choice. To an extreme, the service team can wind up getting too focused on their own internal processes and expenses and completely forget about the customer. This is neither good nor bad, only something that must be decided by the organization. I can do a lot of my household shopping at either Target or Walmart. I’m probably going to have a better experience at Target, however, it will probably cost more than Walmart. What’s most important to you?

Within the enterprise, I’ve seen this dilemma occur in areas where some specialized technical knowledge is needed, but where the services themselves ore not standardized/commoditized. The most frequent example is that of Data Services. Many organizations create a centralized data services group to retain oversight over all SQL written. The problem with this is that while we may have a team that can write great SQL, we may not have a team that really understands what information the business needs from those queries. The team may try to minimize the amount of services available rather than give their customers what they need. What’s the right answer?

When establishing a service team, you need to think about your engagement model. Are you going to provide an outsourcing model, or a consulting model? In an outsourcing model, the service is largely provided without customer input. Customers simply pick from a list of choices and the burden is completely on the service team to provide. Getting a server or a network connection may be much better suited for this category. In a consulting model, there’s a recognition that some amount of input from the customer is still needed to be successful. I can’t create a good data service without knowledge of the customer’s information needs. A consulting model is going to be more expensive than an outsourcing model. If an organization is judging the success of this team based on cost, however, that’s a problem. This is an issue with the success criteria, however. First and foremost, we want to be sure that the right solution gets built. When the services being provided are provided in a cookie-cutter approach, the emphasis can be on cost. When each service requires customization, the focus needs to shift a little bit more toward providing the right solution, with cost as a secondary concern. It may not be about creating reusable services, but usable services that are less likely to cause problems than a custom-built solution by staff without the proper expertise in a given area.

More and more, we’re seeing articles that are now questioning whether SOA is just the next overhyped thing that has come out of IT that has failed to deliver on its promises. Based on a conversation with colleagues at the SOA Consortium, I think at least part of the reason for this is that many organizations still don’t really understand what they’re getting into. At the same time, there’s been a debate on the Yahoo SOA group on SOA as a business thing versus an IT thing. All of this seems to point back to a lack of a clear definition and picture of what we’re talking about.

As I started forming my ideas to blog on this, I was listening to a Software Engineering Radio podcast, which just happened to be a discussion on SOA with Nico Josuttis, who has just authored a book called, “SOA in Practice: The Art of Distributed Systems Design”. In the discussion, I thought Nico put it very well when he stated that SOA is essentially distributed system design where the distributed components have distinct owners. He then said the essentially, there won’t be any projects where a single team has full end-to-end control. It was that second statement that is the key message.

There’s no shortage of organizations that are simply taking the same projects that they’ve been doing for years and simply producing services now as part of them. Is this SOA? It is possible that the project could have been of significant enough scope (more likely a program) to where it included the development of services along with multiple consumers of those services, potentially yielding immediate benefits. The problem with judging SOA from this standpoint, however, is that it is unlikely to scale. Not all projects have this broad of scope. Secondly, it didn’t adhere to the second statement by Nico. By being underneath a common project or program, that effort had complete control over the entire domain. Yes, that project probably had multiple teams involved, but it’s likely that ultimate authority rested with the project architect and project manager, and if they said to do something, it was going to be done that way. Funding decisions and other governance issues don’t even come into play, because it’s all controlled by the project umbrella. An easy litmus test for this is to ask what the organization will do when a new consumer outside of the project comes along and wants to use one of the services that was (or is being) built, but needs some modifications. If it occurs while the project is still in flight, the project manager is likely to raise their hand and say, “Sorry, scope creep, can’t do it.” If it occurs after the project is complete, the new project may have no idea who to even talk to, because the whole structure around that service was based upon the project, and now that the project is over, it’s now being ignored, save for bug fixes/production problems. This isn’t a good situation.

I’m a believer that if you are adopting SOA, you’re committing to a fundamental change in the way IT operates. Of course, this assumes that there are opportunities for improvement within IT. If your IT department is delivering the right things to the business at the right time and for the right cost, great, you’re in the top 5% of organizations and have probably already embraced the changes, whether due to SOA or not. If you’re in the remaining 95% of organizations, you need to give some thought to where you want to go with IT. Is the use of shared services a goal? What are the domains where you know you’ll get some sharing? Is it more about business agility and being able to change the way things are wired in a more efficient manner? How will you tackle the problem of competing priorities when two projects want to leverage a new service, but have competing timelines, or even worse, where one project has control over one consumer and the service, and the other consuming project is at its mercy? How does the organization need to be changed to support this model?

Rather than try to paint my own picture of what this future state could be, I’m going to point all my readers to Nick Malik’s series on Joe Freeflier, part 1, part 2, and part 3. This does a great job in doing this. While this may represent an extreme example, and one that could take 5-10 years to reach, it does a great job in introducing the concepts and ideas that you need to be thinking about as you go forward with SOA.

Pete Lacey had a very interesting post entitled, “What is SOA?” I encourage you to go to his blog and read the whole thing, but I wanted to call out a couple of nuggets here for some comments. First:

“But wait,” I hear my SOA-loving readers say. “SOA is not about exposing business logic on the network. That’s just a technology thing. SOA is about the business! CxOs and business units don’t care about technology, they will only pay for business solutions.” Which always makes me scratch my head. What exactly does IT ever do that’s not about the business? Do they not work for the same company as the other BUs?

This is very reminiscent of the recent discussion on the business case for SOA, which I argued should be a business case for services, not SOA. Pete hits the nail on the head with his comment, “What exactly does IT ever do that’s not about the business?” Of course, it’s probably better stated, “Shouldn’t everything IT does be about the business?” After all, there’s probably some things going on most large IT shops whose business value is debatable. Anyway, what’s really interesting about this is how many activities within IT don’t go far enough to be about the business. IT is just as guilty of putting up walls as the business partners are. Recently, I had a discussion regarding instrumentation of services and collection of usage metrics. I argued that anyone who is acting in the capacity of a service manager should be looking at those metrics on a regular basis. Think about this. How frequently do you run into someone within IT who has recently put a solution into production who is actively monitoring the usage of it to determine if it is a business success? More often than not, I see project teams that quickly dissolve, allocated off to other projects, and no one paying attention to run-time usage unless some red light goes off because something’s broken. This type of monitoring should not strictly be the domain of “the business”. As IT, you are PART of the business, and you should be proactive in your monitoring and understanding of how your solutions are supporting the business. If you throw it over the wall into production, then the business-IT relationship is going to look very similar.

Pete also states:

“Well, it’s not just about business alignment,” another group of SOA advocates claim. “It’s really about governance.” Again, I’m scratching my head here. Everything is about governance! Software development (network-oriented or not) is about governance: ‘You must use a version control system; you must write unit tests.’ Moving systems into production is about governance. Updating a routing table is about governance. Hiring a new employee is about governance. Buying a plane ticket is about governance.

“No, no.” They go on. “With SOA there’s new things to govern.” That’s true, there are. But really, is it that much different than any other governance process? Not really.

I certainly agree that the need for governance is not just limited to SOA. What I’ve found in my work is that long term SOA success requires a fundamental change in the way IT operates. It’s not limited to SOA, however. SOA is merely the trigger that is bringing attention to the flaws in the system. As IT grows, there will be a need for governance, period, whether it’s for SOA or not. This doesn’t mean that SOA isn’t important, but it also means that there’s a risk that SOA becomes the banner under which all things wrong with IT are destined to be cured, and that’s a dangerous path. I’ve done work where the “SOA deliverables” contained a whole bunch of stuff that arguably were not specific to SOA at all. At the same time, if that’s what it takes to bring some attention to some things that need to be fixed, I’m not going to argue. Just be cautious in how much falls under that banner, because you can inadvertently jeopardize your SOA efforts on things that have nothing to do with it.

In a SearchWebServices.com article, Rich Seeley asked the question, “Who owns the SOA business case?” I think this is asking the wrong question. The business case should not be for SOA, but for the services. While some may read this and simply think that I’m nitpicking on semantics, I think it’s an important distinction. If the business case is for SOA, what’s the scope of that SOA? Is it an application architecture? An architecture that encompasses a line of business? An architecture that encompasses the entire enterprise?

The business case should provide the justification for what gets built. The services are what will be built. The article used the term “SOA project” numerous times, which is a pet peeve of mine. I don’t think there should ever be a “SOA project” as you can’t build a SOA. So, the reality is that we have business projects, which probably had an owner assigned the same way that it’s always been done for all business projects.  As part of the project, one or more services are going to be constructed. This equates nicely with Miko Matsumura’s analogy, where he described the interaction between an electrician and a home owner. The electrician likely knows more about the electrical system than the home owner, but the home owner is the one picking the actual light fixtures. Location may be a joint decision, as the home owner may not understanding implications of particular locations. The decision on what goes on in the walls is probably exclusively the responsibility of the electrician (based on the local building code). There are always exceptions. My father was an electrician in the Chicago area, so he and I have a preference for wires flowing through pipe rather than the use of romex, and may choose to have more say in that decision. The same may hold true for a tech-savvy business owner.

Both Tony Baer and Dana Gardner have posted blog entries in response to the article and get closer to the notion of what the right thing to own is. Dana suggests that ownership should reside at the business process level. This is one way of slicing it, but it’s not the only way, and therefore, it’s probably not the right way. In my opinion, ownership comes into play at many levels. A service can be shared across multiple applications. A service can be shared across multiple business processes. If ownership of the shared service is assigned to a single application or business process owner, there’s risk. The right way is to draw a line between business ownership of a service and business ownership of service consumers. You’ll probably need both. Now the risk associated with this approach is the number of individual owners may grow quite large. While I do think that an organization should try to make some determination up front on the potential for a service to be shared, I also know that whatever direction they go, things will change. So, it is important to have an idea on how to change ownership of a service after the fact. Just as a commercial organization may begin with “bundling” and eventually converge multiple products into one, so may be the case with services. If this makes sense to you, then you’re grasping an aspect of the problem that hasn’t been discussed yet, which is, “What does a service owner actually do?” The owner of the business case, and ultimately, the service, is not simply the source of funding and scope decisions for the project that builds version 1.0, but must be the owner of the whole service lifecycle from version 1.0 to the decommissioning of the last version, handling the onboarding of new consumers, “service” to existing consumers, etc.

Two recent posts by Jeff Schneider and Nick Malik, recently brought some attention to a very important aspect of SOA, which is funding it.

Jeff’s post, entitled “Budgeting for SOA” gives a great breakdown of all aspects of adopting SOA, which based on my experience, would certainly be a multi-year effort. He begins with establishing the SOA Foundation, which includes establishing a strategy, roadmap, reference architecture, and governance strategy, continues on with SOA Infrastructure Realization, establishing a SOA Governance Team, performing architectural analysis, training the staff, and then building the services and their consumers. Jeff articulates the costs associated with each of these for a typical organization based upon his experience.

Nick’s post, address the other side of this, which is where to find the money. In his post, SOA Economic Model: Avoiding Chargeback with Transaction Ration Funding, Nick calls out his disdain for chargeback models, and instead presents an alternate view whereby shared resources are funded out of a fixed operating budget that is funded through some form of flat tax. The teams that are funded by this pool are then allocated funds based upon the amount of transactions they process. While Nick presents a very simple model based upon the number of transactions, I’m sure other funding models could be used since a simple request count doesn’t take into account the complexity of the services, but his model is accurate in that it properly incents the service development teams. The teams that use the services are going to pay the tax no matter what, so they’re also now incented to make use of the services being provided, unlike a chargeback model whereby they pass less if they don’t use the shared services.

I think both of these posts do an excellent job of laying out the possibilities. Unfortunately, I wish more organizations were at the state where they have to worry about these factors. It seems that the vast majority of organizations are simply trying to build services out of existing efforts. Unless those efforts are sufficiently broad in scope (and surprisingly, many organizations I’ve seen do have at least one major initiative that typically qualifies), it’s unlikely that services will be created that will have broader applicability. Now I consider myself an optimistic person, but I also recognize that this is not an easy challenge. To get to what I envision and what Jeff and Nick describe represent fundamental changes in the way software development takes place. To do this will mean leveraging staff that already operates out of a shared funding model, such as enterprise architecture, since their efforts merely need to be assigned by the EA manager. If the EA group establishes the appropriate strategic models, smaller projects with limited scope can now demonstrate how the will contribute to broader strategic goals, thus warranting funding of shared services, versus doing things they way they’ve always been done. All of this comes back to something that I frequently bring up in discussions around SOA adoption. If the only thing that’s changed within IT is that the same old projects that we’ve been doing for years now are creating services within them, do we really expect to see any kind of dramatic change within IT, or will it just be chalked up as another overhyped trend? I, as an EA, certainly plan on doing everything I can to make sure that’s not the case. We have to be careful not to bite off more than the organization can chew at any given time, but we also need to make sure that we are the impetus for change.