Todd Biske: Outside the Box

Both Collin Smith and Rob Eamon responded to my post regarding my participation in an upcoming panel discussion at the Gartner EA Summit asking for my thoughts on SOA, EA, and EA Frameworks, so I thought I’d oblige.

First off, I can be considered a “big SOA” advocate. That is, I think it needs to be applied at something larger than a project scope. I think it needs to be an initiative that is not tied to any one particular implementation project. This implies that it needs to be driven by a group in the organization that is not predominantly consumed with project activities. One obvious candidate, therefore, is an Enterprise Architecture organization. In fact, I can’t think of any other organization that is as good of a fit. Individual managers may embrace it, but they are typically not positioned to guide the organizational and cultural changes required, except at the highest levels. Their primary concern (and rightly so) is typically keeping their stakeholders happy. Enterprise-wide, or even department-wide adoption of SOA can be very disruptive in the short term. So, if I had to pick a group to drive SOA adoption, it will almost always be enterprise architecture.

As for whether “SOA folds into EA,” I agree with Rob’s comments. SOA doesn’t replace Enterprise Architecture, it’s simply one view of the enterprise. Today, one could argue that most organizations view the IT landscape as a collection of applications. Efforts like application rationalization or application portfolio management reinforce this notion. So, you could also say that today we have application oriented architectures. The unit of composition is the application. This isn’t flexible enough, as it is too coarsely defined. If we break these applications into smaller uniits, we now get to service oriented architecture, which I feel is a better way of describing things. Is it the only way? Certainly not. There may be value in a process-oriented view. We still need deployment-centric views that simply show physical, and now virtual, servers. We may need a network-centric view. These are all tools in the toolbox of the Enterprise Architecture team, and depending on your specific responsibilities within that team, some may be more important than others. As I’ve mentioned before, I have a background in human-computer interaction going all the way back to my college days, and one thing that I’ve always believed is that it is very unlikely that one view, whether it be a diagram, a user interface, will meet the needs of everyone. This is why I’m also not a huge fan of EA Frameworks. I think EA frameworks can be of great value when you’re starting out. The scope of EA can be daunting, and if you’re tasked with establishing an EA practice in an organization, it never hurts to begin with an established framework. When those frameworks become too focused on trying to make everything fit into a one-sized fits all approach, rather than on actually making the effort successful is where things can become problematic. Within EA, I don’t think it’s necessarily the fault of the frameworks, but more due to EA being an immature practice. While the concepts have been around for more than a decade, there are still many large organizations (at least in the area where I live) that don’t have an EA practice at all, or have only been doing it for 2 or 3 years. While my sample base is relatively small, my experiences have been that every organization does it differently. Some EA groups have significant authority, some have virtually no authority. Some groups spend all of their time engaged on projects, some have no engagement with projects. Some are committees, some are standing organizations. Some are exclusively focused on managed the technology footprint, some are actively involved with business strategy and business architecture. With this much variation, it’s hard for any framework to achieve wide adoption, because they’re simply not a good fit for the short term needs that the EA team needs to accomplish. When the primary artifact of EA tends to be intellectual capital (i.e. thought leadership, future state models, etc.), you need to have flexibility in how that capital is represented, because consumption is the number one factor, not standardization.

I’ll be part of two panel discussions at the upcoming Gartner Application Architecture, Development and Integration and Enterprise Architecture Summits. These are being held at the Rio Casino and Conference Center in Las Vegas the week of Dec. 3-7. In the App Arch summit, I’ll be part of a Power Breakfast discussing funding SOA on Tuesday morning at 7:30 am. In the EA summit, I’ll be part of a panel discussion jointly moderated by Gartner and The SOA Consortium discussing the relationship between EA and SOA on Wednesday at 3:30 pm.  I’ll be at the two summits from beginning to end (Monday – Friday), so feel free to find me and say hi.  One of the more enjoyable parts of these conferences for me is the networking opportunities.

Dan Foody of Progress Software had an interesting blog recently called UDDI in a Web 2.0 World. In it, he asks:

SOA What? With all of this Web 2.0 development, it’s clear that internet scale folksonomies work far better than taxonomies. On the other hand enterprises are, for the most part, stuck with UDDI-related SOA governance tools and their strict taxonomy and categorization mechanisms. The open question though… is this really a problem?

Aside: I love the use of SOA What? That’s exactly why I try to always say S-O-A. On the subject, however, I think Dan raises an interesting question. One of the questions I’ve asked some of the registry/repository vendors is “Can you be indexed by a Google Appliance?” Admittedly, I’m not a huge fan of taxonomy-based searching. At the same time, however, a typical enterprise asset repository may not have enough critical mass to get appropriate metadata for folksonomy based searching. The Web is filled with hyperlinks. How many links to a service detail page am I going to have inside a typical enterprise?

Personally, I’d rather try to find a way to build up the metadata than go crazy building taxonomies to support direct navigation. First off, you can quickly get into taxonomy hell where there are so many variations that you try to support that it becomes difficult to present to the user. Second, people are so used to using Google, Desktop Search, Spotlight, etc. Universal search is going to be a standard part of the office toolset, and we need to find a way to ensure relevant results get returned. This will likely require analysis of software development artifacts (including source code) and building up those relationships based upon presence within project repositories and the role of the user performing the search. A developer performing a search will want to see very different results when searching on “Customer service” than a business manager.

The challenge we face is that the documents and their metadata are scattered all over the place. I previously asked if metadata should be the center of the SOA universe. Neil Ward-Dutton replied that it the center of the universe, and is inherently federated. We need intelligent crawlers that can infer the appropriate relationships and feed this into the universal search engine. Is anyone out there leveraging a Google appliance or other universal search option to facilitate searching for services and other IT assets? If so, like Dan, I’d love to hear about your experiences.

Mike “The Mad Greek” Kavis had an interesting post on SOA Lessons Learned over at IT Toolbox. First off, a big thank you to Mike for sharing some experiences about an effort that didn’t go the way it was originally planned. We can learn as much from these as we can from success stories.

The part of the post that caught my attention began with this line:

As we started the second wave of projects, I mandated that all code should be delivered with test harnesses, the build process should be 100% automated, and testing automation should be part of the project deliverables.

Mike went on to discuss how automation and governance were quickly forgotten when the schedule began to slip. The surprising thing to me was not that these aspects were dropped when the schedule began to slip, but that the implementation of things like continuous integration and testing automation were tied to the SOA effort to begin with.

To me, this was indicative of something that I’ve seen at a number of places which is to use “SOA” as the umbrella to fix all things in need of improvement within the software development process. Continuous integration is a great thing and everyone should be practicing it. But if you organization hasn’t adopted it or created a standard, repeatable way of doing it, don’t target your pilot for another key initiative (like SOA) to try to make it happen. SOA adoption is not dependent on having a continuous integration system. If you have it, will it make SOA easier? Yes, probably, but more so because it’s making all development easier, regardless of whether you’re practicing SOA or not. Give it its own pilot where it can be successful in a very managed fashion, and roll it out to the rest of the enterprise.

Part of the problem is finding ways to adopt these improvements in the development process. Is the business going to care about continuous integration? You can argue that they should, but it’s really about internal IT processes. All too often, IT is left to take the congressional lawmaker approach and find some big project that will be sure to be funded and then push everything but the kitchen sink into it under the radar. This creates additional risk and often results in the project team biting off more than they can chew. It’s unfortunate that IT frequently has little ability to improve on its internal processes due to the project-centric nature of its work. Take another support organization like HR. While I’m sure some work in HR work is project-driven, a lot of the work is day-to-day operations. My suspicion is that organizations that are focused more on fixed-cost day-to-day work like this probably have more ability to take on internal improvement initiatives.

My point of all this is that an organization has to be careful on what they try to take on. There’s always opportunities for improvement, and the point should be quality, not quantity. Trying to take on everything is unlikely to lead to success on any of it. Taking on a smaller set of goals and ensuring that you do them well is a safer approach.

I just read about three blog entries that all had to do with Joe McKendrick’s recent post titled, “Enterprise SOA concept falls out of favor.” Others commenting on this post include Jeff Schneider and this one (author unknown).Personally, I’m tired of these articles that are lamenting SOA. If you read between the lines, the real story is that there are very few case studies of “big SOA” in the way it was originally hyped. Therefore, we need to start hyping “small SOA” to keep it relevant. Neither one of these is the right thing. If “Big SOA” isn’t succeeding, we should be asking why? Odds are, it’s because the organizations involved are simply trying to do IT the same way that they’ve always been doing IT, and merely hoping that by creating some “services” things will suddenly be better. Sorry, won’t happen. In this scenario, what’s really happening is “Small SOA.” Because we haven’t changed anything about the way IT defines its activities, the only hope of achieving any of the hyped goals of SOA is to find some existing projects that are of sufficient enough scope to create that potential. If they don’t exist, now it’s a pure bottom up approach where people are throwing arbitrary services out there and hoping for the best. Certainly having some services is better than no services, but you’d be hard pressed to claim that the organization has adopted SOA and shown value that a CIO would appreciate.Coming back to “Big SOA”, it also doesn’t do any good to say, “it must be enterprise” and try to get funding for some all-encompassing, top-down enterprise initiative. That’s unlikely to happen. What needs to happen is “right SOA.” In order to do it, however, you need to have some knowledge of where you’re going to get the most bang for your buck. I point back to my post on horizontal versus vertical thinking for this. If you don’t have an idea on how to break down your functional capabilities into this, you need to take the time to better understand your business. Do the analysis, and then make the decisions on where to apply it. If you only focus on projects at hand, you may have success on a project with appropriate scope, but then it all falls apart when you don’t have the next target area. This doesn’t have to be some huge, all-encompassing analysis effort, but it does need to be enough to ensure that the techniques you’re going to apply are relevant, will add value, and will be successful. If we don’t have this knowledge, how can we have any confidence that any new technology approach, whether it is SOA, REST, BPM, EDA, AJAX, Web 2.0, etc., will be successful?

I’m a frequent listener of Biotech Nation with Dr. Moira Gunn, available through IT Conversations, and have begun learning more about the world of biotechnology. In doing so, I realized that there are some good analogies between the practices in agriculture and what we need for governance from IT. I’m not the first one to use an agriculture analogy, as Neil Macehiter and Neil Ward-Dutton described the IT technology landscape as a garden in their book, “The Technology Garden: Cultivating Sustainable IT-Business Alignment.”

When we grow something, it starts with the seed. The seed needs to germinate to grow into something, be it corn, wheat, soybean, etc. There’s plenty of things that can go wrong at this stage, so seeds are normally pre-treated with some form of protection, such as an pesticide. Once the seed germinates and the plant begins to grow, the farmer or gardener now needs to worry about weeds and insects. In the past, this involved heavy doses of chemicals, be it pesticides or herbicides. The problem with these is that these chemicals tend to work selectively on particular bugs or particular weeds. If you had a different bug or a different weed, you threw more chemicals on it. Not only is there risk that these chemicals stay on the food that the plants produce, but also that the runoff does harm to the environment. Today, there are non-specific herbicides that will kill just about anything that is green and leafy. The problem with this is that is would normally kill the crop. To that end, biotechnology has allowed crops to be grown that have a resistance to these non-specific herbicides. The net result is that less chemicals are necessary. Rather than continual spraying of selective herbicides, a single spray of a non-specific herbicide can be used.

So how does this relate to governance? The seed is the typical IT project. This is the thing that we’re trying to grow. Pick the wrong seed and you have problems. If you don’t pre-treat the seed, it may never germinate. The theme here is that projects need to be positioned for success from the beginning. There are activities that take place at the inception (or even before) of a project that can make or break your efforts. Given that most IT shops are still very tactical in their activities, SOA adoption is still predominantly a bottom-up effort. If you don’t properly scope your projects, as well as watch it carefully at the beginning so that any service development efforts are properly scoped, your chances of SOA success (or long-lasting IT success for that matter), will be less.

Step two in the growing process was the use of biotechnology. These plants have genetics that create a resistance toward the bad things that could come along. In the IT project sense, this is all about making the right thing the path of least resistance. If you’re successful with doing this through education, tooling, and mentoring, you won’t need to worry about review committees and rigid processes, because your seed will naturally do the right thing.

Finally, there is still a need for some general oversight. Weeds can still grow in your garden, and those weeds can consume vital resources that your plants need to live. In the IT sense, this isn’t about the specific decisions on the projects, it’s about the resources allocated to the project. Even if your architecture is sound, if your resources aren’t empowered to do the right thing, or if they’re pulled in many directions by multiple projects, you’re going to have problems.

So, while many people have frequently used the city planning analogy for architecture and governance, I’m beginning to like the gardening/farming analogy even more. Create an environment that positions things for success, make the “right thing” the path of least resistance for the project, and finally, keep the weeds out, allowing the resources allocated to focus on making the project successful.

For those of you that are seeing “Governing the Garden” in your feed reader, but no post here, it’s coming. I accidentally posted it before I had completed it, then deleted it. For some strange reason, it’s still in the RSS feed. Quirk of WordPress, I guess.

Dana Gardner has posted the latest edition of his Briefings Direct SOA Insights podcast series, which is a discussion on virtualization and IT operations efficiency. Besides myself, we had a big group for this discussion including Jim Kobielus, Neil Macehiter, Dan Kusnetzky, Brad Shimmin, JP Morgenthal, and Tony Baer.

I find both of these topics very interesting and was glad to be part of the discussion. Operational management is an area ripe for improvement in many organizations, and it’s something that doesn’t get a lot of attention, because the impact to the business is largely ignored unless something breaks. Somehow, we need to move from a firefighting mode to a value-add mode where it’s more about the collection of data during normal operations to assist in the continued success of the business.

You can read a full transcript of the discussion here.

It’s about 6:20 in the morning, and I’m presently on a 5 hour bus ride with a bunch of IT staff headed toward some facilities of my employer to learn more about their business. Before I get into the topic for this entry, I certainly want to give kudos to my employer for setting up this opportunity for IT to learn more about the business. With the long bus ride, I’m trying to get caught up on some podcasts. I’m presently listening to a discussion on SOA Management with Jason Bloomberg of ZapThink and Dana Gardner of Interarbor Solutions.

In his intro, Dana Gardner used the oft-mentioned and maligned term, silo. For whatever reason, it suddenly occurred to me that we always work within silos. Organizational structures create silos. Projects create silos. Physical locations can create silos. So, perhaps the right discussion shouldn’t be how to eliminate silos, but rather, how to choose silos correctly, work appropriately within them, and know when to redefine them. I’ve commented a bit on how to organize things, specifically in my post on horizontal and vertical thinking. For this entry, I’d like to discuss the appropriate way to operate within a horizontal silo.

A horizontal silo is one where the services being provided from that silo have broad applicability. A very easy example most organizations should be familiar with is servers. While there are multiple products involved based on the type of processing (e.g. big number crunching versus simple transactional web forms), you’d be hard pressed to justify having every project select its own servers. This area isn’t without change, one only needs to look at the space of VMWare and its competitors to see this.

In many IT organizations, when a centralized group is established, the focus can often be on cost containment or reduction. For many horizontal domains, this makes a lot of sense, but there’s a risk associated with this. When a group focuses on cost reduction, this is frequently done at the expense of the customer. Cost reduction typically means more standardization and less customer choice. To an extreme, the service team can wind up getting too focused on their own internal processes and expenses and completely forget about the customer. This is neither good nor bad, only something that must be decided by the organization. I can do a lot of my household shopping at either Target or Walmart. I’m probably going to have a better experience at Target, however, it will probably cost more than Walmart. What’s most important to you?

Within the enterprise, I’ve seen this dilemma occur in areas where some specialized technical knowledge is needed, but where the services themselves ore not standardized/commoditized. The most frequent example is that of Data Services. Many organizations create a centralized data services group to retain oversight over all SQL written. The problem with this is that while we may have a team that can write great SQL, we may not have a team that really understands what information the business needs from those queries. The team may try to minimize the amount of services available rather than give their customers what they need. What’s the right answer?

When establishing a service team, you need to think about your engagement model. Are you going to provide an outsourcing model, or a consulting model? In an outsourcing model, the service is largely provided without customer input. Customers simply pick from a list of choices and the burden is completely on the service team to provide. Getting a server or a network connection may be much better suited for this category. In a consulting model, there’s a recognition that some amount of input from the customer is still needed to be successful. I can’t create a good data service without knowledge of the customer’s information needs. A consulting model is going to be more expensive than an outsourcing model. If an organization is judging the success of this team based on cost, however, that’s a problem. This is an issue with the success criteria, however. First and foremost, we want to be sure that the right solution gets built. When the services being provided are provided in a cookie-cutter approach, the emphasis can be on cost. When each service requires customization, the focus needs to shift a little bit more toward providing the right solution, with cost as a secondary concern. It may not be about creating reusable services, but usable services that are less likely to cause problems than a custom-built solution by staff without the proper expertise in a given area.

Who says blogging doesn’t pay? In my mail this weekend was a package from none other than Greg, of Greg the Architect fame containing a t-shirt (thankfully in adult sizes and not in action figure sizes) and a letter about his upcoming exploits. Greg is looking to leverage “Social Networking” (as he put it) in the development of future episodes and is looking for the help of the blogosphere. In his exploits to achieve SOA success, he’s dealt with vendors, trade shows, and industry analysts. What new topics would you like to see Greg take on? So far, REST hasn’t come into the mix, nor have we seen much exposure of the internal workings of Greg’s company (does it have a name?). I encourage you to go to Greg’s website and share you ideas. He was very explicit in his letter that credit will be given appropriately!

More and more, we’re seeing articles that are now questioning whether SOA is just the next overhyped thing that has come out of IT that has failed to deliver on its promises. Based on a conversation with colleagues at the SOA Consortium, I think at least part of the reason for this is that many organizations still don’t really understand what they’re getting into. At the same time, there’s been a debate on the Yahoo SOA group on SOA as a business thing versus an IT thing. All of this seems to point back to a lack of a clear definition and picture of what we’re talking about.

As I started forming my ideas to blog on this, I was listening to a Software Engineering Radio podcast, which just happened to be a discussion on SOA with Nico Josuttis, who has just authored a book called, “SOA in Practice: The Art of Distributed Systems Design”. In the discussion, I thought Nico put it very well when he stated that SOA is essentially distributed system design where the distributed components have distinct owners. He then said the essentially, there won’t be any projects where a single team has full end-to-end control. It was that second statement that is the key message.

There’s no shortage of organizations that are simply taking the same projects that they’ve been doing for years and simply producing services now as part of them. Is this SOA? It is possible that the project could have been of significant enough scope (more likely a program) to where it included the development of services along with multiple consumers of those services, potentially yielding immediate benefits. The problem with judging SOA from this standpoint, however, is that it is unlikely to scale. Not all projects have this broad of scope. Secondly, it didn’t adhere to the second statement by Nico. By being underneath a common project or program, that effort had complete control over the entire domain. Yes, that project probably had multiple teams involved, but it’s likely that ultimate authority rested with the project architect and project manager, and if they said to do something, it was going to be done that way. Funding decisions and other governance issues don’t even come into play, because it’s all controlled by the project umbrella. An easy litmus test for this is to ask what the organization will do when a new consumer outside of the project comes along and wants to use one of the services that was (or is being) built, but needs some modifications. If it occurs while the project is still in flight, the project manager is likely to raise their hand and say, “Sorry, scope creep, can’t do it.” If it occurs after the project is complete, the new project may have no idea who to even talk to, because the whole structure around that service was based upon the project, and now that the project is over, it’s now being ignored, save for bug fixes/production problems. This isn’t a good situation.

I’m a believer that if you are adopting SOA, you’re committing to a fundamental change in the way IT operates. Of course, this assumes that there are opportunities for improvement within IT. If your IT department is delivering the right things to the business at the right time and for the right cost, great, you’re in the top 5% of organizations and have probably already embraced the changes, whether due to SOA or not. If you’re in the remaining 95% of organizations, you need to give some thought to where you want to go with IT. Is the use of shared services a goal? What are the domains where you know you’ll get some sharing? Is it more about business agility and being able to change the way things are wired in a more efficient manner? How will you tackle the problem of competing priorities when two projects want to leverage a new service, but have competing timelines, or even worse, where one project has control over one consumer and the service, and the other consuming project is at its mercy? How does the organization need to be changed to support this model?

Rather than try to paint my own picture of what this future state could be, I’m going to point all my readers to Nick Malik’s series on Joe Freeflier, part 1, part 2, and part 3. This does a great job in doing this. While this may represent an extreme example, and one that could take 5-10 years to reach, it does a great job in introducing the concepts and ideas that you need to be thinking about as you go forward with SOA.

Pete Lacey had a very interesting post entitled, “What is SOA?” I encourage you to go to his blog and read the whole thing, but I wanted to call out a couple of nuggets here for some comments. First:

“But wait,” I hear my SOA-loving readers say. “SOA is not about exposing business logic on the network. That’s just a technology thing. SOA is about the business! CxOs and business units don’t care about technology, they will only pay for business solutions.” Which always makes me scratch my head. What exactly does IT ever do that’s not about the business? Do they not work for the same company as the other BUs?

This is very reminiscent of the recent discussion on the business case for SOA, which I argued should be a business case for services, not SOA. Pete hits the nail on the head with his comment, “What exactly does IT ever do that’s not about the business?” Of course, it’s probably better stated, “Shouldn’t everything IT does be about the business?” After all, there’s probably some things going on most large IT shops whose business value is debatable. Anyway, what’s really interesting about this is how many activities within IT don’t go far enough to be about the business. IT is just as guilty of putting up walls as the business partners are. Recently, I had a discussion regarding instrumentation of services and collection of usage metrics. I argued that anyone who is acting in the capacity of a service manager should be looking at those metrics on a regular basis. Think about this. How frequently do you run into someone within IT who has recently put a solution into production who is actively monitoring the usage of it to determine if it is a business success? More often than not, I see project teams that quickly dissolve, allocated off to other projects, and no one paying attention to run-time usage unless some red light goes off because something’s broken. This type of monitoring should not strictly be the domain of “the business”. As IT, you are PART of the business, and you should be proactive in your monitoring and understanding of how your solutions are supporting the business. If you throw it over the wall into production, then the business-IT relationship is going to look very similar.

Pete also states:

“Well, it’s not just about business alignment,” another group of SOA advocates claim. “It’s really about governance.” Again, I’m scratching my head here. Everything is about governance! Software development (network-oriented or not) is about governance: ‘You must use a version control system; you must write unit tests.’ Moving systems into production is about governance. Updating a routing table is about governance. Hiring a new employee is about governance. Buying a plane ticket is about governance.

“No, no.” They go on. “With SOA there’s new things to govern.” That’s true, there are. But really, is it that much different than any other governance process? Not really.

I certainly agree that the need for governance is not just limited to SOA. What I’ve found in my work is that long term SOA success requires a fundamental change in the way IT operates. It’s not limited to SOA, however. SOA is merely the trigger that is bringing attention to the flaws in the system. As IT grows, there will be a need for governance, period, whether it’s for SOA or not. This doesn’t mean that SOA isn’t important, but it also means that there’s a risk that SOA becomes the banner under which all things wrong with IT are destined to be cured, and that’s a dangerous path. I’ve done work where the “SOA deliverables” contained a whole bunch of stuff that arguably were not specific to SOA at all. At the same time, if that’s what it takes to bring some attention to some things that need to be fixed, I’m not going to argue. Just be cautious in how much falls under that banner, because you can inadvertently jeopardize your SOA efforts on things that have nothing to do with it.

While doing my thinking outside the box, I ran across Joe McKendrick’s post discussing whether SOA can be boxed. Joe provided commentary on a blog entry from Jack van Hoof who called out that the major ERP-vendors could potentially deliver “SOA in a box.” He points out:

SAP offers a service bus, service registry, events registry, canonical data management, business processes, services deployments (!), business monitoring, business process management, security… out-of-the-box. Yes, of course the implementation must be tuned and configured. But it’s all there, out-of-the-box.

There’s a certain amount of truth to this statement. It’s probably better stated as services in a box, because after all, it’s a box. We have no idea what the underlying architecture of SAP is. While it may be the case that SAP or any other large ERP system could constitute the bulk of your infrastructure, that doesn’t mean that you’re suddenly purchasing an SOA. If new business needs come along, it’s now up to your ERP system to provide those capabilities. If they’re new, then it’s the ERP vendor who must figure out how to quickly make the changes necessary, if they even choose to do so, since it would have to be something with broad customer applicability to make financial sense for them. It’s certainly possible to build custom code on top of the ERP system, but you’re always going to have that dependency. That’s not necessarily a bad thing, you just have to choose wisely on where to leverage the ERP system.

I’d like to pick on one comment Joe made in his commentary. He stated:

…the idea of buying all SOA from one vendor flies in the face of the ultimate meaning and purpose of SOA — the ability to pick and choose tools, applications and services from any and all vendors. SOA is supposed to mean the end of vendor lock-in.

I don’t agree with this opinion. While services can be used to create an abstraction layer from vendor products, I don’t think it needs to be a goal. The factors that influence whether an organization leverages one vendor, lots of vendors, or no vendors really doesn’t come into play at all. What SOA should do is assist you in making appropriate vendor decisions. Just as I commented some time ago that SOA should neither increase nor decrease outsourcing, but instead ensure increase the chance that outsourcing efforts are successful, the same holds true for choosing vendors. By breaking the problem domain down to a finer-grained level (services rather than applications), I can make better decisions on the vendor products I choose. If they don’t expose services for the capabilities I need, I’m going to look elsewhere. The only thing that could start leading toward better insulation from vendor lock-in will be more standards in the vertical domains. There’s plenty of standards out there, but there’s probably far more spaces that are not standardized.

So, what’s my advice? I don’t think you can buy “SOA in a box,” you can only buy “services in a box.” Your enterprise architects need to be the ones defining the architecture, and then leveraging the architecture to ensure that not only your home grown systems, but also your vendor systems, whether from one or many, adhere to it.

Neil Ward-Dutton of Macehiter Ward-Dutton had a post recently entitled “Rethinking IT projects? Think service, not product, focus.” He called out a frequent mantra of mine, which is that we need to get away from the typical project-based mentality and toward a product-based mentality. While Neil agrees that we need to get away from the project-based mentality, he questioned whether a product-based mentality is the right approach either. Instead, he advocates a service-based focus, where “service” in this case is more akin to its use in ITIL and IT Service Management (my interpretation, not his).

If you dig into his post, and if you’ve been following my own posts, I think you’ll find that we’re essentially saying the same thing, but perhaps using different terminology. Neil states:

The trouble is that taking too literal a view of IT delivery through the lens of product management can prevent you from reflecting reality the way that “customers” (regular business people in your organisation, and quite possibly those external customers that ultimately pay all the salaries) see it.

I couldn’t agree more. I’m a huge advocate for usability, user centered design, etc., so anything that involves assumptions on what the user needs rather than going out and actually involving the customer is definitely red flag in my book. Personally, I don’t even like to use the term customer when talking about IT delivering solutions to the business where the business is the end user. The business and IT should be partners in the effort, not a customer/supplier relationship.

Futher in the post, Neil makes the comment:

If you take too much of a product management centric view, the danger is that you focus all your energy creating the right kind of development and deployment capabilities, without thinking of the broader service experience that customers need and expect over the lifecycle of a long-term commitment. IT operations is where the rubber meets the road, and where customer expectations are met or dashed. Too simplistic a focus on product-style management for IT delivery perpetuates the development-operations divide and squanders a great opportunity.

Personally, I don’t view what Neil describes as good product management. If your definition of product management is simply a chained sequence of development activities, you’re missing the boat. Only through excellent operational management and customer interaction can you make appropriate decisions on what those subsequent activities should be. It’s not all about development. I think this point was made very well by Dr. Paul Brown in his book, “Succeeding with SOA: Realizing Business Value Through Total Architecture” which I was fortunate enough to review with Dana Gardner. He made very clear that projects are only successful when they deliver the promised business value. This value doesn’t come when the software is deployed into production. It comes after it. It may be one month, it may be one year. I’ve been involved with a number of projects that defined the project mission statement in terms of “delivering something by such-and-such date for $$$.” While meetings dates and budget are very important, they don’t define success. Delivering business value defines success, and the effort needs to continue to ensure that happens even if there are no development activities occurring. This brings in Neil’s notion of service, and it is absolutely a critical component to success.

In a SearchWebServices.com article, Rich Seeley asked the question, “Who owns the SOA business case?” I think this is asking the wrong question. The business case should not be for SOA, but for the services. While some may read this and simply think that I’m nitpicking on semantics, I think it’s an important distinction. If the business case is for SOA, what’s the scope of that SOA? Is it an application architecture? An architecture that encompasses a line of business? An architecture that encompasses the entire enterprise?

The business case should provide the justification for what gets built. The services are what will be built. The article used the term “SOA project” numerous times, which is a pet peeve of mine. I don’t think there should ever be a “SOA project” as you can’t build a SOA. So, the reality is that we have business projects, which probably had an owner assigned the same way that it’s always been done for all business projects.  As part of the project, one or more services are going to be constructed. This equates nicely with Miko Matsumura’s analogy, where he described the interaction between an electrician and a home owner. The electrician likely knows more about the electrical system than the home owner, but the home owner is the one picking the actual light fixtures. Location may be a joint decision, as the home owner may not understanding implications of particular locations. The decision on what goes on in the walls is probably exclusively the responsibility of the electrician (based on the local building code). There are always exceptions. My father was an electrician in the Chicago area, so he and I have a preference for wires flowing through pipe rather than the use of romex, and may choose to have more say in that decision. The same may hold true for a tech-savvy business owner.

Both Tony Baer and Dana Gardner have posted blog entries in response to the article and get closer to the notion of what the right thing to own is. Dana suggests that ownership should reside at the business process level. This is one way of slicing it, but it’s not the only way, and therefore, it’s probably not the right way. In my opinion, ownership comes into play at many levels. A service can be shared across multiple applications. A service can be shared across multiple business processes. If ownership of the shared service is assigned to a single application or business process owner, there’s risk. The right way is to draw a line between business ownership of a service and business ownership of service consumers. You’ll probably need both. Now the risk associated with this approach is the number of individual owners may grow quite large. While I do think that an organization should try to make some determination up front on the potential for a service to be shared, I also know that whatever direction they go, things will change. So, it is important to have an idea on how to change ownership of a service after the fact. Just as a commercial organization may begin with “bundling” and eventually converge multiple products into one, so may be the case with services. If this makes sense to you, then you’re grasping an aspect of the problem that hasn’t been discussed yet, which is, “What does a service owner actually do?” The owner of the business case, and ultimately, the service, is not simply the source of funding and scope decisions for the project that builds version 1.0, but must be the owner of the whole service lifecycle from version 1.0 to the decommissioning of the last version, handling the onboarding of new consumers, “service” to existing consumers, etc.