Todd Biske: Outside the Box

While I’m on the subjects of podcasts, another good listen was this presentation from Tim Brown, CEO of Ideo, on Innovation Through Design Thinking which was presented at the AdaptivePath conference. One of the great comments he made was how innovators take in everything from the environment around them to get new ideas. This hit home, because that’s one of the ways I go about writing this blog. I simply take in all the information around me, and sometimes things click. A great example of this was my blog on the power of feedback loop which came to mind after watching a video on a cow-powered dairy farm. If you consider yourself an innovative thinker, give it a listen.

I’m here at the airport listening to the latest IT Conversations Technometria podcast from Phil Windley and company. This one is a conversation with Phil Wolff of Skype Journal. If you have any interest in VoIP and Identity Management, this is a great listen.

Some of the blogs I follow have been simply giddy about the recent statements on REST from Anne Thomas Manes of the Burton Group. About the only thing I agree with on some of the comments is that Anne is extremely smart. Beyond that, this is a case of people seeing what they want to see. Of course, some will accuse me of doing the same, but that’s okay. All of our blogs are simply forums for stating our own opinions, so that’s what it should be!

First, let’s call out where the debate exists. The debate has been around REST versus WS-*, not REST vs. SOA or ROA (Resource Oriented Architecture) vs. SOA. Even there, you could narrow it even more, which was eloquently captured by Erik Johnson in a comment on Stefan Tilkov’s blog (which came to my attention courtesy of Don Box’s blog).

“It seems to me that people attracted to REST (in whatever form) are rebelling against interface-based programming more than WS-* itself — at least that’s my excuse.”

There have been endless debates in the blogosphere and mailing lists over whether or the message payload is part of the interface or not. Clearly, REST is all about having a uniform interface, and if you include message semantics in that interface, it seems that it would be a difficult thing to achieve. The uniform interface is GET, PUT, POST, and DELETE. There’s been some recent discussion about media types and their role in implying message semantics, however, the WS-* proponents will argue that there’s a huge gap between application/xml and having XML Schema (courtesy of WSDL).

Anyway, back to the statements from Burton Group. While most of the bloggers are commenting on this news story at SearchWebServices.com, I first caught wind of this directly from the source via the Application Platform Strategies Blog at Burton Group. My take on this is that Anne first recognizes that REST isn’t going away anytime soon, as is increasing in mindshare. No arguments here, and there are plenty of REST bloggers out there popping champagne bottles over this statement.

The piece that I feel is getting lost in some of this discussion is that, as Anne points out in the APS blog:

REST is not the same as “plain old XML” (POX). POX refers to the format of a message payload, and it says nothing about architectural style. It just says that you don’t wrap an XML message with an XML envelope (e.g., SOAP or Atom). More to the point, not all POX applications are RESTful, and not all RESTful applications are POX.

This is probably the biggest myth right now. There are lots of people that are using XML over HTTP, thinking that they are using REST, when in fact, they aren’t. Again, Anne nails it with this statement:

REST is not simply technology–it’s an architectural style that’s fundamentally different from they way most developers design systems today. It requires a noun-oriented approach to designing systems rather than one based on verbs. I know quite a few people that have been studying REST for years who still struggle with RESTful design practices. Understanding the basics of the style is easy. Truly groking it and being able to apply it to real-world situations is much harder.

This is where I’d like to see some constructive conversations. When we’re talking about SOA, we’re talking about services. Services represent capabilities. Capabilities are typically associated with verbs. REST is resource-driven. To me, that would mean that I should apply it where I need a resource oriented architecture, not a service oriented architecture. The question is where is this appropriate? Clearly, there are many calls that do nothing more than retrieve data, and it would seem that a REST-based approach for this would work very well. The question, however, is whether a resource oriented view is sufficient.

One of the things I like about SOA is that it can help establish better lines of ownership, which theoretically, can allow IT to operate more efficiently. Because these are based on services, however, they are more likely to be aligned along functional boundaries rather than on resource boundaries. Resources are shared across functional boundaries, so if my unit of composition is at a resource level, how do I deal with these concerns? I’m not suggesting that it can’t be done, but I think this is where there is lots of room to grow, as Anne points out. In the SearchWebServices example, Anne provided a light bulb example, which (pun intended) should turn on the light bulb for people on what they’re getting into:

“A REST application to turn on and off the lights in your building will require you to design a URI for every light bulb and then you send it on/off messages,” she explained. “It’s not like I have a single service that manages all my light bulbs. It’s a very different approach to designing a system. And it’s going to be really hard for developers to get their hands around it.”

So what is my opinion on all of this? If it’s doesn’t come across from my blog, I tend to very pragmatic. I think there are places for both WS-* and REST, and that will continue for the foreseeable future. REST makes lots of sense to me when we’re dealing with browser-based clients (e.g. AJAX). WS-* makes lots of sense to me for service-to-service interactions. I do fall more on the side of formal interfaces, and as a result, I want to learn more about WADL. I’ve yet to see a solution that is doing REST by the book, most examples I’ve been fall more into the POX/HTTP category, or using an HTTP GET with query parameters to return data as XML (all read only). That doesn’t mean they don’t exist, I just haven’t seen them. In any case, debates such as these keep things interesting. There’s always risks that it will strictly be a battle of religions rooted in opinions (which never get resolved). Involvement of people like Anne Thomas Manes and others that fall into the pragmatic group in the middle can ensure that the debate progresses to appropriate application of these approaches to where we leverage the strengths of both, and minimize their use in areas where the weaknesses are exposed.

I was recently having a conversation regarding data services and my recent posts (here and here) on horizontal versus vertical thinking put it in a new light. My experience has shown me that data services are one of the most controversial topics within an enterprise.

Nearly all solutions, somewhere along the line, are going to need to access some form of structural data, typically from one or more relational databases. The debate around data services begins with a discussion on whether or not all access to the data should go through a service or not. I’m not going to get into that part of the debate in this post. The point of this discussion lies more with the organizational challenge of creating a data services layer.

One reason (but certainly not the only one) that is sometimes used to justify a data services layer is to maintain control over how the databases are used. All it takes is one bad SQL statement to cause a significant impact to all systems using that database. When organizations see this problem, the instinct is to centralize the development of data services. By creating a team with expertise in database access technologies, the assumption is that you’ll get high quality services that will prevent system failures due to data unavailability. The challenge with this approach is that this assigns ownership and service boundaries along a technical domain: data access. Per my previous postings, assigning ownership based on technical boundaries implies a horizontal domain. Horizontal domains are ones where the capabilities are becoming commodities and can be done in a standard fashion. Service teams in a horizontal domain typically strive to minimize the number of services involved. This is where the challenge occurs. Data access is anything but a commodity. While the techniques used to access data may be standardized (SQL), the structure of data and the combinations of data desired by consumers is certainly not.

In his book Human Interactions,
Keith Harrison-Broninski pointed out that everyone deals with information differently. We file it differently, we take notes differently. Trying to standardize information access is inherently problematic. Why is this? Because information retrieval operates in a vertical domain. We’re retrieving information for the purpose of doing a particular task. So, we have a services team that is viewing the world in a horizontal fashion, with their primary goal being to minimize the number of services they provide. On the other side of the equation, we have vertical thinking consumers that want data for a particular purpose in a format that is optimized for what they need to do. Clearly, if the consumers have the most influence, the goal of the service team to minimize the number of services is not going to happen. If the service team has the most influence, the goal of the consumers to have optimized services is not going to happen. We simply have a situation where the two (or more) teams involved have competing goals. You can’t say that either one is right or wrong, only that they are competing with each other. There can be sound justification on both sides, so the organization must mediate and try to create a win-win situation.

I think this subject is a great example of the concepts of horizontal and vertical thinking. Things can go very smoothly when both the consumer and provider are consistent with their understanding of a particular domain as either horizontal or vertical. When there is disagreement, and one group sees things as horizontal while another sees things as vertical, the relationship is going to be problematic. If you can make it a win-win situation, that is the best scenario. If one party or the other simply has a view that is inconsistent with the view of enterprise architecture, that’s a communications and education issue for EA. EA needs to set the policies for what constitutes good architecture, and if a particular functional domain is deemed to be horizontal (or vertical) that information should be well communicated so the individual solution architect approaches its usage properly.

James McGovern had an interesting post recently entitled “Are Business Applications Boring?” It reminded me of some of my own experiences, both as a supervisor and as an individual. A few years ago, a group that I oversaw was very focused on Java middleware technologies. .NET was gaining in prominence at that time, and it was clear to me that the team would need to gain Microsoft experience, and it was entirely likely that the majority of our work in our future would be on the .NET platform. I told the team that they needed to determine what was more important to them: writing successful Java solutions or writing successful business solutions. In addition, I asked myself that same question. To me, I was more interested in seeing the organization and the business be successful than I was about writing Java code or C# code. For others, they chose to part ways with the company as the amount of Java work reduced. There’s absolutely nothing wrong with that, although, I do think there’s more long term risk for a developer in going that path. Why? It’s far easier to find someone who can write code than to find someone who understands the business and the culture.

Interestingly, I left the enterprise world about 9 months ago and joined the consulting ranks. That being said, if you’ve followed my blog, you’ll know that I don’t view SOA as a technology initiative. Proper application of technology in support of business needs is far more of a business issue than it is a technology issue, and that’s what interests me. My view on SOA consulting is that it needs to be focused on business consulting more so than technology consulting. This subject came up in a recent conversation with Dana Gardner and his analyst gang that will be published soon.

The short of this is that it is difficult for enterprise IT practitioners to hold on to top technical talent unless those individuals are interested in the business. If I were to go back to school today, I would pursue an MBA, not additional technology education. If individuals are solely focused on the technology, they are unlikely to get long term satisfaction from a 30-plus year career at one organization. Unfortunately, I haven’t seen very many developers who are passionate about understanding the business. I can’t remember the last time that I interviewed someone and had them ask me some serious questions about the business to see whether it would be a good fit, when you’d think that would be one of the most important factors.

In my mind, the writing is on the wall. If you’re in the early stages of your career and want to put yourself on a path for long term growth, you’ll need to build up deeper and deeper knowledge of the domains where you apply technology. If you just want to maintain a status quo, you can become a developer, however, I suspect that your salary will remain part of the status quo, as well.

James McGovern made the following comments regarding my Service Adverse Architecture post:

Todd Biske is right on the money by echoing the fact that companies who have mastery of SOA also have forward thinking management. I wonder if him and Joe McKendrick would define a litmus test so that others can characterize their own enterprise in terms of the management team?

While I’ll have to give some thought to the entire litmus test, the first thing that immediately came to mind was trust, and it’s not limited to management. If I’m a project manager, I want to have as much control over my own destiny as possible. When I now become dependent on other groups and other projects outside of my control and my management to be successful, that’s a big leap of faith. Unfortunately, it’s part of human nature to remember the one project where a team had problems rather than on the times the team was successful. As a result, parties may go into the relationship with the expectation that someone is going to screw up, spend all their time looking for it ready to point blame, and as a result, that’s exactly what happens. If the groups instead focused on what is necessary to be successful, they’d probably be successful.

We live in a culture where making a mistake is unacceptable. One only needs to look at the current political process to understand how much our culture is based on avoiding failure rather than achieving success. A legislator is not allowed to change a stance on any issue, even though we as individuals do it all the time because we learn more as we go along. Cultures that are based on avoiding failure are, in general, going to have a difficult time doing more than maintaining the status quo. Innovating and advancing means taking risks, and when you take risks, some of them don’t pan out. Teams should not be penalized when they don’t pan out, unless it’s clear that they made very bad decisions based upon the information available at that time. I can look back at many decisions I’ve made in the past and recognize that some of my assumptions didn’t come true. As long as I’ve learned from that and don’t repeat those same decisions, there should be no penalty associated with it.

I suspect that organizations that are more forward-thinking have a greater level of trust within the organization. There is more collaboration than competition, and people all understand that everyone in the organization has the best interests of the organization as a whole in mind, not the best interests of themselves, their team, or their manager. Furthermore, management works effectively with the individuals in the trenches to help them understand what levels of risk the organization will tolerate and what the boundaries for innovation are within each group in the organization (essentially around roles and responsibilities). I’d much rather have an application developer creating innovative applications rather than trying to leverage a new Java framework. At the same time, the team responsible for development frameworks needs to be open to receiving input from the application developer. If they have mutual trust, the development framework team will be open to hearing about new alternatives, and the application developer will not throw a fit if the development framework team decides that it’s not in the best interests of the company to leverage the new framework at this time.

SOA will create more moving parts associated with the delivery of IT solutions. More moving parts means more ownership and hence, more interaction among teams. If we don’t trust each other, the chances of success are greatly reduced. That being said, trust must be earned and maintained, it can not be established by edict. Service providers must do all they can to build trust. Management must ensure that the organization takes an “innocent until proven guilty” approach, rather than the opposite, with actions that back it up.

I just downloaded Microsoft’s Office Open XML Converter beta for MacOS X, and was surprised to see that it was a 24.9 MB Disk Image. The application that was installed was only 3.9 MB, however, I assume that there is a bunch of supporting files that went into the Application Support folders somewhere. As a point of comparison, the freely available docx-converter widget for the Dashboard downloads at 154 KB and post-installation, comes in at 304 KB. While I’m confident that the Microsoft converter is more robust, I’m really surprised at just how big it was.

I just listened to Elizabeth Book’s podcast with Anne Thomas Manes of Burton Group and Joe McKendrick discussing Service Averse Architecture. They definitely echo statements that I and others have been making for some time. Particular quotes that I want to highlight:

The companies that are good with SOA are likely to be … companies that have forward-thinking management and are good at everything else, as well … The organizations that really could use SOA, that need that flexibility and agility, are the ones not likely to be adopting SOA.
– Joe McKendrick

Joe and I both had posts on this topic back in October of 2006 (see here and here). This is simply the nature of way things progress. You have a certain class of organization that are the leaders that have a high percentage of success at whatever they do. The problem, however, comes with the middle class. What we’d like to see is that middle class of organizations merely trailing the leaders, but still eventually achieving success. Instead, we’re at risk of seeing the “haves” and the “have nots.” Anne calls out that Service Averse Architecture is the status quo, and that’s a shame. As long as it continues to be the status quo, we’re not going to progress as an industry, and if anything, it fuels the fire of those that question the relevance of IT. After all, if businesses continue to thrive despite having continued dissatisfaction with IT support, then is IT really a differentiator worth investing in, or is it simply a necessary cost center where the focus should be on driving costs down to rock bottom levels? I believe IT can be a differentiator, but like Anne, I do believe it’s a very select few who are able to do so.

The same type of thinking does hold true in other business support areas. Is HR simply a necessary evil, or can it be a differentiator? Look at Southwest Airlines hiring practices and the experience they are able to give their customers and you can see the impact. Despite having no reserved seats, their customer satisfaction (and profitability) is far better than most airlines. Thankfully, a few shining stars can go a long way in keeping efforts alive to ensure that we don’t have the “technology haves” and the “technology have nots.” It’s why I am supportive of efforts like the SOA Consortium.

The big issue is that most organization have a culture and an incentive system that absolutely discourages the adoption of good SOA practices. That is, everything is project focuses and every project is completely focused on delivery of that project as quickly as possible at the least possible cost and there is no interest whatsoever in figuring out, “well, what is it that I’m building that should be implemented as a service that other people can use?”
– Anne Thomas Manes

I was happy to hear Anne call out the problems with the project-based culture prevalent in IT organizations that I also commented on in April of 2007. There are huge cultural changes that need to be made, but most organizations are not doing so. It’s an extremely difficult problem, because you need to figure out an incremental way of doing it. Anne goes on to comment:

So much of SOA is tied up in things like Master DM and ID M and a reasonable DA and BPI efforts and all the EA things people are doing. You have to take this cross-organizational perspective and plan what you’re going to do and not just say, “Oh, I’ve got a new project I’m going to let my developers figure out what services to build.” It’s challenging because you have to think globally but you do actually have to implement on a local level. But you just can’t depend on the local people to figure out what they’re supposed to be doing. They need guidance from above.
– Anne Thomas Manes

This articulates the challenge well. You can’t simply say tell the developers to build the right services, even if you’re allowing them to challenge the scope boundaries of the projects, without having appropriate guidance from above. I’ve see a number of efforts where developers are honestly trying to do the right thing, but they don’t know the right next steps to take. The project-based culture is so ingrained that even when they are able to identify appropriate services that have broader value, they don’t know what to do to ensure that those services actually see broad use. Too often, it’s simply a “let’s build it and hope they come” approach, where the only guaranteed consumer is that initial project that kicked it all off. In short, not much has changed.

I talk about this as a lifestyle. If you want to become healthy and fit you need to exercise and eat right. Well, if you want your systems to be agile and flexible, then you have to adopt SOA fitness activities. You have to recognize what’s good behavior, what’s bad behavior, and try not to do the bad things, try to do the good things.
– Anne Thomas Manes

If you’ve been lucky enough to have someone really think about what it means to adopt SOA, you’ll understand exactly what Anne is saying here. SOA winds up touching all aspects of IT. If your doctor tells you that you are at risk of a heart attack, simply eating less fatty foods is not enough. There is no magic pill or single activity that will make your SOA adoption efforts successful. It is a lifestyle, and you need to be committed to everything that it may entail, rather than only doing it where it is convenient. We have to recognize that to make things better for the business, things have to change. If we’re only implementing minor changes, we can only expect minor successes. If your IT department is already very successful, minor changes may be fine. If your IT department is on a path to a heart attack, major changes may be in order.

Update: As usual, the timing of Dilbert is perfect. The Dilbert for Monday, May 22nd really captures the status quo and how there is far too much competition rather cooperation that occurs within the enterprise.

Joe McKendrick posted a followup with some of the discussions that have went on within the comments of his blog regarding his commentary on the recent ZapFlash about barriers to SOA adoption. While most of the comments amount to vendor bashing, the one that stuck out to me was the one that said, “SOA offers nothing different than what the vast majority of good IT teams have been doing for more than a decade. Think strategically? We do. We recognize SOA for what it is: nothing.”

My first comment is that SOA is indeed, evolutionary, not revolutionary. That being said, I simply don’t believe that most IT departments can truly believe that they’re doing a great job. Are there some that are? Absolutely. Are those great ones thinking strategically? Probably so. But what about the rest of the pack? To them, I ask one simple question: What has changed? If you’re viewing this strictly as a technology thing where you throw some REST or Web Services into the same-old solutions that you’ve been building, you’re not getting it. If the business is unhappy with IT, something needs to change. Perhaps its a change in how things get built and how we view lifecycles (such as product lifecycles instead of development lifecycles). Maybe that manifests itself in organizational changes. Perhaps it is more operationally focused with visibility and reporting on subsystems that are of significance to the business rather than another IT worker. These are just some examples. If you’re still defining projects in the same old way, deploying them in the same old way, etc., well then, not much has changed, has it. I hope that 10 years from now some things have changed, but IT moves very slowly. 10 years ago, I was hoping usability and user-centered design philosophies would continue to gain prominence, and those are still in their emerging stages in many corporate enterprises. At least I know I’ll have a long career in advocacy!

I just read Eric Newcomer’s writeup on Gregor Hohpe’s JavaZone presentation on InfoQ. I haven’t watched the presentation yet, but it’s in my to-do list, and Eric’s writeup certainly raised the priority of it a notch.

Eric makes a lot of good points in his writeup, but the most important one in my opinion, is his last statement which I wanted to call out here:

The point of SOA is not to make life easy for the developer of a service – it’s to make life easy for the consumer of the service – which might (and probably should) be someone else.

Well said Eric!

John Wu commented on my horizontal and vertical thinking post earlier in the week, and I felt it warranted a followup post. He stated:

Enterprise Architecture is a horizontal thinking while application development is a vertical thinking.

While I understand where John was coming from, I think that this approach is only appropriate at the very early stages of an EA practice. The first problem that an organization may face is that no one is thinking horizontally. This may go all the way down to the infrastructure level. Projects are always structured as a top-to-bottom vertical solution. While there may be individuals that are calling out some horizontal needs, unless the organization formally makes it someone’s responsibility to be thinking that way, it will have difficulty gaining traction.

Unfortunately, simply creating an EA organization that thinks horizontally and still letting the project efforts think vertically is not going to fix the problem. If anything, it will introduce tension in the organization, with developers claiming EA is an ivory tower and EA claiming that developers are a bunch of rogues that are doing whatever they want in the name of the project schedule.

If we characterize where the organization needs to go, it’s where both EA and the development organization are thinking both vertically and horizontally. This does come back to governance. Governance is about decision making principles to elicit the desired behavior. The governance policies should help an organization decide what things are horizontal, what things are vertical, and then assign people to work on those efforts within those architectural boundaries. Right now, many organizations are letting project definitions establish architectural boundaries rather than having architectural boundaries first, and then projects within those boundaries. Project boundaries are artificial, and end when the project ends. Architectural boundaries, while they may change over time, should not be tied to the lifecycle of a project.

So, EA should be concerned with both the vertical and the horizontal nature of IT solutions. Based upon the corporate objectives, they should know where it is appropriate to leverage existing, horizontal solutions and where it is appropriate to cede more control (although maintaining architectural consistency) to a vertical domain. Two systems that have some redundant capabilities but are architecturally consistent at least create the potential for a consolidation at some later point in time, when the corporate objectives and policies change. In order to do this, the project staff must also be aware of both the vertical and horizontal nature of IT solutions.

I’m currently reading The Technology Garden: Cultivating Sustainable IT-Business Alignment by the Neils from Macehiter Ward-Dutton along with Jon Collins and Dale Vile of Freeform Dynamics. In the spirit of full disclosure, the publicist sent me a free advance copy of the book, as Amazon reports its availability as June 11th. I’m about halfway through it, and it’s been a pleasant read. Chapters four and five have particularly caught my attention. They are titled, “Create a common language” and “Establish a peer relationship between business and IT.” The common language chapter puts the onus on IT to learn the language of the business. While they also state that no competent business executive should be technology-ignorant (my words, not theirs), the bulk of the burden is on the technology staff. In the next chapter, it begins with a discussion on how many IT groups play a supplier role, and how that simply isn’t good enough these days. While service delivery and management is very important for building trust, it’s not sufficient. They state:

Suppliers, by definition, do what they’re told. The customer is always right! The parameters of service delivery are defined by the ‘customer,’ and thereafter the supplier delivers, in response to requests, in the context of those parameters (you can think of these as ‘contracts’ and ‘service-level agreements’).

When I read this, it occurred to me that as an industry, we really haven’t made much progress on the whole concept of IT and business as peers. I’ve mentioned previously that in my early days, I did a lot of work on user interface technologies, and had a strong interest in human-computer interaction during college. My first introduction to user-centered design techniques and viewing the end user as a partner in the process was in the summer of 1993. That was 14 years ago, and yet I’d have to say that in general, IT still operates in a supplier role, with things thrown back and forth over the business/IT wall. I really liked the emphasis on communication in Chapter 4 of the book. If the continued prevalence of IT as a supplier mentality is due to a fundamental lack of trust, the only thing that will eventually break it down is communication. I’ve certainly been guilty of falling into the typical technologist mode of communicating: email. As they call out, it’s time to starting getting out of our chairs, out of our comfort zones, and start communicating. If you don’t know who to begin your conversations with, seek someone out that can help you with that.

I had the experience of participating in an exercise directed by a CIO where we were split into groups of 8 people, and then furthered subdivided into two groups of 4 seated at separate tables. Each table had a task to accomplish as outlined on a piece of paper. All tasks were identical, and each set of 8 people had identical equipment at their table to complete the task. The goal of the task was to ensure that each sub-group of 4 built identical solutions. If you’ve seen Apollo 13, this whole thing was prefaced by a video clip from the movie where the engineers at Mission Control dump a box full of stuff on the table and have to figure out a way to build a carbon dioxide scrubber out of it. They then have to get the Apollo 13 astronauts to do the same. The interesting thing about this task was that the instructions were not very limiting. For example, there was nothing that said one group of four couldn’t get up and go sit with the other four and complete the whole thing together. The point of the exercise was that we set many artificial boundaries in our work based on past experiences, culture, etc. In fact, there are probably more artificial boundaries than real boundaries. Does your company have a stated policy that you can’t go and talk to an end user on the business side? If they don’t, there’s nothing preventing you from doing that. If we’re going to begin building trust back up in the IT/Business relationship, it is time to step outside of the box and start communicating as peers. Let’s not wait another 14 years to start making a change.

While I’ve never used Ruby on Rails, I caught wind of these from Richard Monson-Haefel’s blog and found them pretty funny. They’re in the spirit of the Mac versus PC ads.

• Ruby on Rails vs. Java
• Ruby on Rails vs. PHP (part 1)
• Ruby on Rails vs. PHP (part 2)
• Ruby on Rails vs. PHP (part 3)

Update:Added the link for the fourth and final video in the series.

Alastair Bathgate is taking the SOA Consortium to task in a response to Joe McKendrick’s recent post about the goals of the SOA Consortium. The stated goals of the SOA Consortium, as Joe correctly states, are to have 75% of the Global 1000 complete a successful SOA implementation by 2010.

Alastair asks, “What on earth is the measure of success?” This type of questioning is certainly appropriate, and probably the biggest hurdle that the SOA Consortium will need to overcome. Joe McKendrick calls this out as well, stating that success with SOA is a long-term gain, and that “the only true measures of long-term success in the market are either increased revenues or increased stock values, and many factors besides SOA will contribute to this. The real issue is figuring out how to measure SOA’s contribution to this success.”

For the record, I am a member of the SOA Consortium. The first conference call I was on did include discussion on how to assess SOA success, so I have confidence that this will be addressed. Personally, I think the greater concern is Joe’s final comment. What is SOA’s contribution to the success? Already in my own efforts, there are plenty of things that SOA adoption can call attention to, such as having solid Security and Management architectures or standardized infrastructure practices, none of which are necessarily tied to SOA, but can certainly make life more difficult for SOA adoption. If success is measured in terms of the company’s stock price, can you really say whether SOA had anything to do with a fall or climb? Financial statements are always interesting, because the raw numbers give some quantitative insight, but the qualitative analysis is somewhat of a free-for-all, and highly subjective to the views of both the company’s PR group as well as the analysts.

Alastair expressed some concern about ‘Gorilla Vendor sponsored “clubs”‘ and that’s certainly within his right. Personally, I’m a glass half-full kind of guy, and always look for the good in these things. I think an SOA advocacy group is a good thing. It’s a targeted opportunity for group discussion, similar to a BoF session at a conference. I’ve learned a lot in these environments, and they haven’t been dominated by vendors pushing their will. Maybe I’m the exception, but to each their own. Everyone has to judge the value they see in these efforts and decide accordingly. In any case, I’m glad Alastair is paying attention and blogging on the subject, as it will only help the broader community in the long run.

So, rather than lamenting that SOA success is difficult to determine, what are your thoughts on how to measure SOA success? Comment or trackback with your thoughts. I’ll dedicate an upcoming post to my own thoughts, as well.

I’ve been meaning to post on this subject for some time, so it’s good that I got to the airport a little earlier than normal today. There’s nothing like blogging at 5:30 in the morning.

As I mentioned in my last entry, I just finished listening to a podcast from IT Conversations on the drive to the airport which was a discussion on user experience with Irene Au, Director of User Experience for Google. One of the questions she took from the audience dealt with the notion of having a centralized group for User Experience, or whether it should be a decentralized activity. This question is one that frequently comes up in SOA discussions, as well. Should you have a centralized service development, or should your efforts be decentralized? There’s no right or wrong answer to this question, however, it’s certainly true that your choices can impact your success. In the podcast, Irene discussed the matrixed approach at Yahoo, and how everything would up being funded by business units. This made it difficult to do activities for the greater good, such as style guides, etc. The business unit wanted to maximize their investment and have those resources focused on their activities, not someone else’s. Putting this same topic in the context of SOA, this would be the same as having user-facing application teams developing services. The challenge is that the business unit wants that user-facing application, and they want it yesterday. How do we create services that aren’t solely of value to just that application. At the opposite extreme, things can be centralized. Irene discussed the culture of open office hours at Google, and how she’ll have a line of people outside her office with their user experience questions in hand. While this may allow her to maintain a greater level of consistency, resource management can be a big challenge, as you are being pulled in multiple directions. Again, putting this in the SOA context, the risk is that in the quest for the perfect enterprise service, you can put individual project schedules at risk as they wait for the service they are dependent on. These are both extreme positions, and seldom is an organization at one extreme or the other, but usually somewhere in the middle.

In trying to tackle this problem, it’s useful to think of things as either horizontal or vertical. Horizontal concepts are ones where breadth is the more important concern. For example, infrastructure is most frequently presented as a horizontal concern. I can take a four CPU server and run just about anything I’d like on it these days, meaning it provides broad coverage across a variety of functional domains. A term frequently used these days is commodity hardware, and the notion of commoditization is a characteristic of horizontal domains. When breadth becomes more important that depth, there’s usually not many opportunities for innovation. Largely, activities become more focused on reducing the cost by leveraging economies of scale. Commoditization and standardization go hand in hand, as it’s difficult to classify something as a commodity that doesn’t meet some standard criteria. In the business world, these horizontal areas are also ones that are frequently outsourced, as all companies typically do it the same way meaning there is little room for competitive differentiation.

Vertical concepts are ones where depth is the more important concern. In contrast to the commoditization associated with horizontal concerns, vertical items are ones where innovation can occur and where companies can set themselves apart from their competitors. User experience is still an area where significant differentiation can occur, so most user-facing applications fall into this category. Business knowledge, customer experience (preferably at a partnership level to have them involved in the process), are keys at this level.

By nature, vertical and horizontal concerns are orthogonal and can create tension. I have a friend who works as a user experience consultant and he once asked me about how to balance the concerns that may come from a user experience focus, clearly rooted in the vertical domains with the concerns of SOA, which are arguably focused on more horizontal concerns. There’s no easy answer to this. Just as the business must make decisions over time on where to commoditize and where to specialize, the same holds true for IT. Apple is a great example to look at, as their decision to not commoditize in their early days clearly resulted in them being relegated to niche (but still relevant) status in computer sales. Those same principles, however, to remain more vertically-focused with tight top-to-bottom controls have now resulted in their successes with their computers, iTunes, Apple TV, the iPod, and the forthcoming iPhone. There are a number of ways to be successful, although far fewer ways than there are to be unsuccessful.

When trying to slice up your functional domains into domains of services, you must certainly align it with the business goals. If there is an area of the business where they are trying to create competitive differentiation, this is probably not the best area to look for services that will have broad enterprise reuse, although it is very dependent on whether technology plays a direct role in that differentiation or an indirect role, such as whether the business to consumer interaction is solely through a website, or if it is through a company representative (e.g. a financial advisor). These areas that are closest to the end user are likely to require some degree of verticality to allow for tighter controls and differentiation. That’s not to say they own the entire solution, top to bottom, however, which would be a monolith.

As we go deeper into the stack, you should look for areas where commoditization and standardization outweighs the benefits of customization. It may begin at a domain level, such as integration across a suite of applications for a single business unit, with each successive level increasing the breadth of coverage. There is no point where the vertical solutions stop, and everything beneath it has enterprise breadth. Rather, it is a continuum of decreasing emphasis on depth and increasing emphasis on breadth. A Internet company may try to differentiate themselves in their end-user facing systems that the users interact with, allowing a large degree of autonomy for each product line. The supporting services behind those user interfaces will increase in the breadth of their scope, but still may require some degree of specialization, such as having to deal with a particular region of a country or even the world for global organizations. We then bleed into the area of “applistructure” and solutions that fall more into the support arena. A CRM system will have close ties to the end-user facing sales portal. The breadth of CRM coverage may need to span multiple lines of business, unlike the sales portal, where specialization could occur. Going deeper, we have applications that may have no ties to the end-user facing systems, but are still necessary to run a business, such as Human Resources. Interestingly, if you’re following my logic you may be thinking that these things should all be outsourced, but the truth is that many of these areas are still far from being commoditized. That’s because they involve user facing components, which brings us back to those vertical domains where customization can add value. An organization that outsources the technology side of HR, but doesn’t have an associated reduction in HR staff may have a potential conflict when they want to have specialized HR processes, but are dealing with commodity interfaces and systems. Put simply, you can’t have it both ways.

The trend continues on down the stack to the infrastructure and the world of the individual developer. If you’re truly wanting to adopt SOA from top to bottom, there should be a high degree of commoditization and standardization at this level. Organizations where solutions are still built top-to-bottom, with customized hardware platforms, source code management, programming languages, etc. are going to struggle with SOA, because their culture is vertically-oriented to an extreme.

While the speed of change, business decisions on what things are core competencies and what things are not do not change overnight. Taking an organization where each product group had its own staff (vertically-oriented) and switching it to a centralized sales organization (horizontally-oriented) is a significant cultural change, and often doesn’t go smoothly. You only need to look at the number of mergers and acquisitions that have been deemed successful (less than 50%) to understand the difficulty. Switching from vertically-focused IT solutions to horizontally-focused IT solutions is just as difficult, if not more difficult. Humans are significantly more adaptable than technology solutions, which at the core, are binary, yes/no environments. The important thing is to recognize when misalignment is occurring and take action to fix it. That’s all about governance. If users are trying to apply significant customization to a technology area that has been deemed as a commodity by the business, push back needs to occur to emphasis that the greater good takes precedence over individual needs. If IT is taking far too long to deliver a solution in an area where time to market and competitive differentiation is critical, remove the barriers and give that group more control over the solution, at the expense of broader applicability. If you don’t know what your priorities and principles are, however, for each domain, you’ll end up in and endless sequence of meetings that are rooted in opinions, rather than documented principles and behaviors desired by the organization.