Todd Biske: Outside the Box

Archive for the ‘Enterprise Architecture’ Category

Joe Hill from EDS is on the stage now. The title of his talk is on SOA and Outsourcing, although it’s far more broad than just an outsourcing discussion. It would be nice to have a copy of his slides, as he had some really good ways of visualizing SOA adoption along a timeline and the differences on how it should be approached. For example, using a chart reminiscent of Clayton Christensen’s “The Innovator’s Solution”, he showed that at the early phases where we have “performance undersupply,” the typical solution may need to be more tightly coupled, fewer vendors, and more use of proprietary techniques. The discussion around SOA is typically on the right hand side of the chart, emphasizing loose coupling, multiple vendors, open standards, etc. The problem that I felt Joe was emphasizing was that you need to recognize where you are and apply the right techniques at the right time, rather than on focusing too heavily on the end state on the right hand side of the diagram.

Rob High of IBM is on the stage now with a presentation titled “SOA Foundation” which runs the gamut of topics associated with SOA. One thing he took a lot of time to discuss was the notion of coherency and the importance of semantics to SOA. It was nice to hear some emphasis on this point, as I believe that an understanding of the semantics is a critical component in moving from SOA applied to applications to SOA applied to the enterprise. Just as with the rest of SOA, make sure you understand the semantics first before throwing any semantic technology at it. While there are evolving specs and tools in this space, none of that will do you any good if don’t first understand the semantics themselves and how that information can be leveraged in your project efforts.

I’m here at The Open Group Enterprise Architecture Practitioner’s Conference, and will try to blog where I can on the presentations. Right now, I’m sitting in David Linthicum’s keynote, which is on EA & SOA. He had an interesting quote, which was:

Five years from now, we won’t be talking about SOA… It will all be folded back into EA.

There’s some truth to this, but there’s also a lot of risk. One of the issues with EA (and many other efforts within IT), is that it can become disconnected from the project efforts that are going on. The term most frequently used with this is “ivory tower” where enterprise architects are simply viewed as paper pushers that know how to create a lot of PowerPoint slides. One of the side benefits of SOA is that it relates very well to the world of the development projects, with “service” being the point of common language. The enterprise architects may be modeling the enterprise in terms of the services that are needed, and projects can now utilize these models in their project architecture. This is easier said than done, however, as you need to ensure that the reference material containing these models (e.g. a reference architecture) is consumable at the project level. If the only group that can understand your models are fellow enterprise architects, that’s a problem.

So, will SOA be folded into EA as a whole? If EA can ensure that the artifacts it creates are consumable at the project level, then absolutely, SOA will be folded into EA. If EA is not creating artifacts that are consumable at the project level, then we have a problem. You’ll likely still have tension between EA and SOA, and likely not achieve the levels of success that organizations that have successfully bridged the world of EA and the project space. This doesn’t apply solely to SOA. This applies equally to any architectural domain. You may have information architects working on canonical or enterprise data models, performing data quality analysis, etc., but if it doesn’t find a way to become relevant to project efforts, it will exist on an island with continual struggles to achieve the objectives that were set out.

Joe McKendrick recently had a post regarding bottom-up approaches to SOA in response to this post from Nick Malik of Microsoft, which was then followed up with another post from Nick. I was going to post solely on this discussion, when along came another post from James McGovern discussing bottom-up and top-down initiatives. While James’ post didn’t mention any of the others, it certainly added to the debate. Interestingly, in his post, James put SOA in a “bottom-up” list. Out of this, it seems like there are at least three potential definitions of what bottom-up really is.

Joe’s discussion around “bottom-up” has a theme of starting small. In his followup post, Nick actually concurs with using this approach to SOA, however, he points out that this approach is not what he considers to be “bottom-up.” For the record, I agree with Nick on this one. When I think of bottom-up in the context of SOA, I think of individual project teams building whatever services they want. This puts the organization at great risk of just creating a bunch of services. While you’re guaranteed that your services will have at least one consumer by simply identifying them at the time they’re needed within a project, this is really the same way we’ve been building systems for years. If you change the approach to how you build the solution from the moment the service is identified, such as breaking it out as its own project with an appropriate scope increase to address broader concerns, you can still be successful with this approach, but it’s not easy. When I think of top-down, like Nick, it also starts to feel like a boil the ocean approach, which is probably at just as big of a risk of being unsuccessful.

Anyway, now bringing James’ post into the discussion, he had given a list of three “top-down” IT initiatives and three “bottom-up” initiatives. Top-down ones were outsourcing, implementing CMMi and/or PMI, and Identity Management within a SoX context. The bottom-up initiatives were SOA, Agile Methods, and Open Source. By my interpretation, James’ use of the terminology refers more to the driving force behind the effort when mapped to the organization chart. His top-down efforts are likely coming from the upper echelons of the chart, while the bottom-up efforts (although SOA is debatable) are driven from the lower levels of the chart. I can certainly agree with this use of the terminology, as well.

So is there any commonality? I think there is and it is all about scope. The use of the term top-down is typically associated with a broad but possibly shallow focus. The use of the term bottom-up is typically associated with a narrow but possibly deep focus. What’s best? Both. There’s no way you can be successful with solely a bottom-up or top-down approach. It’s like making financial decisions. If you live paycheck to paycheck without any notion of a budget, you’re always going to be struggling. If you develop an extensive budget down to the penny, but then never refer to it when you actually spend money, again, you’re always going to be struggling. Such is the case with SOA and Enterprise Architecture. If you don’t have appropriate planning in place to actually make decisions on what projects should be happening and how they should be scoped, you’re going to struggle. If the scope of those projects is continually sacrificed to meet some short term goal (typically some scheduling constraint), again, you’re going to struggle. The important thing, however, is that there does need to be a balance. There aren’t too many companies that can take a completely top-down approach. Even startups with a clean IT slate are probably under such time-to-market constraints that many strategic technology goals must be sacrificed in favor of the schedule. It’s far easier to fall in the trap of being overweighted on bottom-up efforts. To be successful, I think you need both. Enterprise architects, by definition, should be concerned about breadth of coverage, although not necessarily breadth of technology as specializations do exist, such as an Enterprise Security Architect. An application architect has narrower breadth than an enterprise architect, but probably more depth. A developer has an even narrower breadth, but significant depth. The combination of all of them are what will make efforts successful both in the short term and in the long term.

It’s funny how these syndicated feeds can be just like syndicated TV. I’ve decided to leverage Google Analytics and create a post with links to the most popular entries since January 2006. My blog isn’t really a diary of activities, but a collection of opinions and advice that hopefully remain relevant. While the occasional Google search will lead you to find many of these, many of these items have long since dropped off the normal RSS feed. So, much like the long-running TV shows like to clip together a “best of” show, here’s my “best of” entry according to Google Analytics.

• Barriers to SOA Adoption: This was originally posted on May 4, 2007, and was in response to a ZapThink ZapFlash on the subject.
• Reusing reuse…: This was originally posted on August 30, 2006, and discusses how SOA should not be sold purely as a means to achieve reuse.
• Service Taxonomy: This was originally posted on December 18, 2006 and was my 100th post. It discusses the importance and challenges of developing a service taxonomy.
• Is the SOA Suite good or bad? This was originally posted on March 15, 2007 and stresses that whatever infrastructure you select (suite or best-of-breed), the important factor is that it fit within a vendor-independent target architecture.
• Well defined interfaces: This post is the oldest one on the list, from February 24, 2006. It discusses what I believe is the important factor in creating a well-defined interface.
• Uptake of Complex Event Processing (CEP): This post from February 19, 2007 discusses my thoughts on the pace that major enterprises will take up CEP technologies and certainly raised some interesting debate from some CEP vendors.
• Master Metadata/Policy Management: This post from March 26, 2007 discusses the increasing problem of managing policies and metadata, and the number of metadata repositories than may exist in an enterprise.
• The Power of the Feedback Loop: This post from January 5, 2007 was one of my favorites. I think it’s the first time that a cow-powered dairy farm was compared to enterprise IT.
• The expanding world of the “repistry”: This post from August 25, 2006 discusses registries, repositories, CMDBs and the like.
• Preparing the IT Organization for SOA: This is a June 20, 2006 response to a question posted by Brenda Michelson on her eBizQ blog, which was encouraging a discussion around Business Driven Architecture.
• SOA Maturity Model: This post on February 15, 2007 opened up a short-lived debate on maturity models, but this is certainly a topic of interested to many enterprises.
• SOA and Virtualization: This post from December 11, 2006 tried to give some ideas on where there was a connection between SOA and virtualization technologies. It’s surprising to me that this post is in the top 5, because you’d think the two would be an apples and oranges type of discussion.
• Top-Down, Bottom-Up, Middle-Out, Outside-In, Chicken, Egg, whatever: Probably one of the longest titles I’ve had, this post from June 6, 2006 discusses the many different ways that SOA and BPM can be approached, ultimately stating that the two are inseparable.
• Converging in the middle: This post from October 26, 2006 discusses my whole take on the “in the middle” capabilities that may be needed as part of SOA adoption along with a view of how the different vendors are coming at it, whether through an ESB, an appliance, EAI, BPM, WSM, etc. I gave a talk on this subject at Catalyst 2006, and it’s nice to see that the topic is still appealing to many.
• SOA and EA… This post on November 6, 2006 discussed the perceived differences between traditional EA practitioners and SOA adoption efforts.

Hopefully, you’ll give some of these older items a read. Just as I encouraged in my feedback loop post, I do leverage Google Analytics to see what people are reading, and to see what items have staying power. There’s always a spike when an entry is first posted (e.g. my iPhone review), and links from other sites always boost things up. Once a post has been up for a month, it’s good to go back and see what people are still finding through searches, etc.

Brandon Satrom posted a blog last week asking the question, “Is Enterprise Architecture Declarative or Imperative?” In clarifying the question, he pointed out that declarative programming is where a developer instructs a system what to do, but leaves it to the system to decide how best to implement that instruction. Imperative programming is where the system is told both the what and how. He goes on to discuss that his thoughts are that EA should be declarative, but working closely with a group of Solution Architects (much like I presented in my People! Policies! Process! post) that take the visions and translate them into execution.

I will agree that for the most part, EA is a declarative activity, although I wouldn’t go so far as to say that the path to execution falls to solution architects. If EA’s merely define the future state, but don’t define the executable actions (albeit at a higher level, they’re not providing project plans), they’ll likely become an ivory tower. Personally, I think the model of increasing constraints is probably a more accurate depiction of what goes on. Through reference models and architectures, there’s a certain set of constraints established that projects must now adhere to. Within the project, there are additional constraints that are established based upon the solution architecture, and then the class design, etc. You could argue that each set of constraints represents a declaration of what things should be.

As soon as you establish constraints, the subject of governance now comes up. I had an interesting discussion with someone today regarding this topic and he made the point that the projects that are intended to operate within the constraints established by EA must be transparent. Rather than assign someone from the security force to watch over the project team with an iron fist, simply make the architectural decisions visible so that at any point, someone can take a look. This made perfect sense to me. I’ve worked at a company that had taken a similar approach for financial governance on projects, where reports were generated every week and the budget watchers could easily determine whether things were compliant or not. It struck me that this would be a good approach for architectural governance, as well. Of course, this does imply that there needs to be a way to actually see the architecture from the outside, which is another story. Visibility to the source code does not equate to visibility of the solution architecture. What are your thoughts on this?

There was a question on the Yahoo SOA group on the notion of SOA compliance, jokingly introduced in this blog from January of 2006 discussing SOA compliant toasters and cell phones. The important question was whether there is a notion of “SOA compliance” or not.

I fall in the camp of saying yes. There won’t be any branding initiative associated with it, however, because it’s not about compliance with some general notion of what SOA is (such as having a product that exposes Web Services or REST interfaces), it’s about a solution’s ability to fit within the SOA of its customers.

It is certainly true that many enterprises are moving toward a “buy” strategy instead of a “build” strategy. This doesn’t mean that they don’t have any work in planning out their SOA, however. Organizations that simply accept what the vendors give them are going to have difficulty in performing strategic architectural planning. The typical IT thinking may be that this is simply an integration problem, but it’s deeper than that.

Previously, I had a post on horizontal and vertical thinking. In this discussion, I called out that an organization needs to understand what domains of functionality are horizontal, and what domains are vertical. When looking at vendors, if their model doesn’t match up with the organization’s model, conflict will occur. It may not be at an integration level, it may be at a higher semantic level where the way the business wants to use a tool just doesn’t match with the way the vendor chose to provide those capabilities. Having a domain model of the business functionality provides a way to quantitatively assess (or at least get a step closer to it) the ability of a product to be successful in the enterprise. This is the notion of SOA compliance, in my opinion.

On the vendor side of the equation, the vendor needs to be able to express their capability in the form of functionality model that can easily demonstrate how it would fit into a company’s SOA. This applies whether it’s a hosted solution or a shrink-wrapped product. Any vendor that’s using SOA in its marketing efforts but can’t express this should be viewed with caution. After all, architectural compliance is only one factor in the equation, however I think it’s one that can have strategic, rather than short-term implications.

In my experience, neither side is very well positioned to allow a quantitative assessment of SOA compliance. Both parties lack the artifacts necessary, and there certainly aren’t any standards around it. Even just having Visio diagrams with rectangles labeled “such-and-such service” would be an improvement in many cases. More often than not, the vendors win out for a simple reason: they have a marketing department, whereas the typical enterprise architecture organization does not. You can assume that a vendor has at least created some collateral for intelligent conversations with Enterprise Architects, however, you can’t assume that an enterprise has created collateral for intelligent conversations with vendors. It needs to go both ways.

I previously posted some thoughts on the new book from the Neils at MacehiterWard-Dutton along with Jon Collins and Dale Vile from Freeform Dynamics called “The Technology Garden: Cultivating Sustainable IT-Business Alignment.” I finally got a chance to pick it up again, and I thoroughly recommend chapter 8, “Foster relationship with key IT suppliers” for anyone who has an involvement in vendor decisions. Working with vendors is an art, and it’s not just the department head who needs to do it. They give some practical guidance on how to determine who your strategic vendors are, and who the vendors are that you simply deal with on a transaction by transaction basis. In addition to this, they also give prescriptive guidance on working with them, and the art of the win-win solution.

In addition to being of value to vendor relations, anyone adopting SOA should also understand this chapter. Why? Because if you’re a service provider, a lot of the same conflicting pressures occur between consumer and a provider. Strive for the win-win situation, and you’ll be better off. It’s not easy to do, especially when you have multiple consumers each with their own priorities.

Simon Brown, on his blog “Coding the Architecture,” had a good, short post on tactical solutions. One of the points he made that I liked was this:

For me, a tactical solution can be thought of as a stopgap. It’s an interim solution. It’s something potentially quick and dirty. It satisfies a very immediate need. Importantly, it has a limited lifespan.

It’s the last sentence that hits home. If you’re calling something tactical, you’d better have a date on the books as to when the solution will be replaced. That’s not easy to do, especially considering how long the typical IT project takes and when funding decisions are made. I had one project where I went to my supervisor and asked him, “Are you okay with paying $$$ for this now, knowing that we will replace it in 18 months to 24 months?” He said okay, and while the real timeline wound up being about 36 months, the replacement eventually did happen. That’s probably the closest I’ve come to truly having a tactical solution. At the time the solution was put in place, a placeholder was put in place for the strategic solution and the process for obtaining funding began.

This brings up a great point for the domain of enterprise architecture. Many architects out there spend a lot of time establishing a “future state architecture.” Often, the future state is a pristine of view on how we want things to be. What doesn’t happen, however, are statements regarding everything that exists today. I’m a big fan of making things as explicit as possible. When I’ve worked on future state architecture, I always ask about the current platforms and what will happen to them. Either they’re an approved part of the future state (and they can be classified as not available for new solutions), or there needs to be a project proposed to migrate to something that is approved. What you don’t want to do is leave something in limbo where no one knows whether it’s part of the architecture or not. The same holds true for so-called tactical solutions. Unless there’s an approved project to replace it with the strategic solution, it’s not tactical, it’s the solution. Don’t let it linger in limbo.

I’m listening to Dana Gardner’s SOA Insights podcast here at the airport. This episode is a discussion with The Open Group about their Enterprise Architect certification. One of the initial questions was whether the role of the SOA Architect was any different from the role of the Enterprise Architect. It seemed that the group in the discussion all agreed that an SOA Architect was a subset of the overall Enterprise Architect role. What did surprise me, however, was the debate on what the responsibilities of the SOA Architect are.

In retrospect, I shouldn’t have been surprised as the same debate is prevalent in Enterprise Architecture as a whole. I’ve worked with organizations whose view on Enterprise Architecture was that it was strictly a technology play, focusing on standard infrastructure. In the SOA space, this is concerned with connectivity between consumers and providers, dealing with the nebulous world of the enterprise service bus, SOA appliances, web services management systems, etc. as well as the technology used to build and execute services, including application servers, orchestration engines, and the like. While less common (but certainly not uncommon), I’ve also encountered organizations where Enterprise Architecture was much closer to the business, including tasks such as business process modeling. Here, it’s much more about the application of technology, rather than the technology itself. Once again, looking at this from the SOA space, the focus would be on the services themselves, not necessarily on the technology used to expose them. What’s my opinion? Simple, it’s both. The only debate is on what is the appropriate organizational structure is for these roles and responsibilities. There’s always some subdivision of responsibilities across multiple people. A topic such as middleware technologies has enterprise applicability but sufficient breadth within it given the continuing evolution of development technologies such that you can have a full time architect focused on the strategy around it.

Regarding whether EA should also include the application of the technology, this was a nice lead in to a second point of debate, which was whether a developer background or a business analyst background leads to an EA career path. Again, I think the right answer is both. A business analyst background will pay significant dividends toward the component of EA concerned with the application of technology, although it depends on how much your analysts participate directly in the technology application efforts. While I do think there’s more potential for some with a technical background to become an EA, simply because of the breadth of solutions to which technologies like Java EE or .NET can be applied, there’s far more value in combining that with business knowledge. Furthermore, the further you go on the business side, the more likely that the role of strategic planner for the business is already being covered by someone on the business side. Those people need to be actively working with EA.

The final point of discussion was around the role of education in Enterprise Architecture. The panelists all agreed that experience was a critical factor, and that you can’t teach experience, however there are aspects that can be taught. One panelist gave the example of a course on TOGAF. The one subject that didn’t come up in this or the discussion around the career path to EA was the ability to be a big picture thinker. For me, this is a must have for anyone in the EA role. Enterprise architecture is about the enterprise. If you can’t see the forest for the trees, you’re going to struggle. If you look at the typical IT organization, what’s the percentage of people who are working on project efforts that are narrowly defined? I’d argue that it is well over 50%. Take out the support and overhead activities, and you’re not left with much. Therefore, EA has to fill that strategic planning gap. You need to be comfortable in working in areas where boundaries are fuzzy, if they’re even known at all. Conflicting priorities and politics are the norm. Personally, I think being a big picture thinker is extremely difficult to teach. I think people naturally prefer to focus on either the details or the bigger picture. I focus on the big picture. As an example, my wife and I are redecorating our front room. She painted everything, and then went out and bought a bunch of artwork for it. She and her mother were hanging the artwork and she asked me to come up and offer my opinion on whether the piano should be moved six inches to the left or not. I told her that it all depended on where the artwork was going to be placed and the balance of the room. I couldn’t offer an opinion solely on the piano, I could only offer an opinion whether considering the entire room. I think I even said that moving the piano six inches was insignificant. Someone who is detail oriented probably would have obsessed over those six inches.

There are risks associated with being a big picture thinker, and it’s important that your EA staff recognize it. Strategy is useless unless you execute. This doesn’t necessarily mean going deep into the details, but it does mean that you need to come up with a logical sequence of activities that can realize the strategy. With each activity, however, you’re narrowing the scope so that you can now leverage the other 90% of the organization that is focused on refining scope and making it happen.

Jordan Haberfield of Excel Partner recently posted a blog titled “Enterprise Architects as ‘Impact Players'”. I’ve enjoyed Jordan’s blog as it discusses EA from a different perspective, one of talent and corporate culture, rather than the technical aspects. I’ve always found the human and social aspects of things very interesting.

Anyway, he provides an excerpt from a book he read called “Don’t Send a Resume: And Other Contrarian Rules To Help Land A Great Job” by Jeffery Fox which introduces the role of the impact player. Here’s the quote:

“There is always a job in a good organization for an impact player. An impact player is someone who can quickly improve the economics of a company. An impact player is someone who can bring in customers, energize the sales force, restructure an under-performing department, speed up the innovation process, solve the late shipment problem, or physically move the manufacturing facility to a lower cost area. An impact player also is someone who will do the necessary but noxious tasks no one else wants to do. An impact player is someone who will get their hands dirty, pick up a shovel and start shoveling, open the store early and close late, deliver product on their way home, deal tirelessly with irate customers and make a service call on Christmas Eve. Good executives in good companies want to hire impact players.”

Jordan goes on to state that “Enterprise Architects are in a position to become that impact player and make a significant difference.” It’s probably better stated that Enterprise Architects should be in a position to do this. Whether they are or not is highly dependent on the corporate culture.

I’ve spoken with a number of colleagues in the St. Louis area, and it’s my understanding that many of the organizations here where I reside don’t even have a formal EA group. Why is this important? It’s important because I believe that an impact player often has to transcend boundaries and operate outside of the constraints that a particular job description may imply. If an organization doesn’t have an Enterprise Architecture discipline, someone needs to go outside of the box of their current job description and start doing EA. If the corporate culture is resistant to people operating outside of their formal job description, that impact player is going to need some very thick skin.

A great example is from two years ago when Jason and Ron at ZapThink emphasized the need for the SOA Champion to guide an organization through SOA adoption. SOA is not about buying an ESB, Registry/Repository, or any other tool. For the bulk of companies that comprise the “status quo” of Service Averse Architecture, it’s about a fundamental change to the way IT solutions are delivered which can cover organizational change, process change, and technology change. What organization has a formal role established for this position? Probably not many. It requires someone to be an impact player, think outside the box, transcend boundaries, and pave the path for the new way of doing things.

Some companies encourage individuals to think outside of the box and outside of their formally stated responsibilities, and it’s probably one that should be added to the litmus test for a company likely to be successful with innovation and strategic initiatives. After all, the degree to which a company does this is a matter of trust. Unfortunately, it’s far easier to break down that trust that it is to build it up. For those of you dealing with that, follow this link to another excellent book that I’ve read.

In summary, the original quote by Dr. Fox states that “There is always a job in a good organization for an impact player.” The key part of this is “good organization.” If you are exhibiting the qualities of an impact player, but you’re struggling to be successful, take a look at the organization’s culture and see whether it is a “good organization” or not. If you’re looking for a good organization, you may need to look for the foot-in-the-door opportunity, because often times, it takes an outsider to come in and recognize the areas for potential impact, so you’re not going to find it on Monster or Dice.

Some of the blogs I follow have been simply giddy about the recent statements on REST from Anne Thomas Manes of the Burton Group. About the only thing I agree with on some of the comments is that Anne is extremely smart. Beyond that, this is a case of people seeing what they want to see. Of course, some will accuse me of doing the same, but that’s okay. All of our blogs are simply forums for stating our own opinions, so that’s what it should be!

First, let’s call out where the debate exists. The debate has been around REST versus WS-*, not REST vs. SOA or ROA (Resource Oriented Architecture) vs. SOA. Even there, you could narrow it even more, which was eloquently captured by Erik Johnson in a comment on Stefan Tilkov’s blog (which came to my attention courtesy of Don Box’s blog).

“It seems to me that people attracted to REST (in whatever form) are rebelling against interface-based programming more than WS-* itself — at least that’s my excuse.”

There have been endless debates in the blogosphere and mailing lists over whether or the message payload is part of the interface or not. Clearly, REST is all about having a uniform interface, and if you include message semantics in that interface, it seems that it would be a difficult thing to achieve. The uniform interface is GET, PUT, POST, and DELETE. There’s been some recent discussion about media types and their role in implying message semantics, however, the WS-* proponents will argue that there’s a huge gap between application/xml and having XML Schema (courtesy of WSDL).

Anyway, back to the statements from Burton Group. While most of the bloggers are commenting on this news story at SearchWebServices.com, I first caught wind of this directly from the source via the Application Platform Strategies Blog at Burton Group. My take on this is that Anne first recognizes that REST isn’t going away anytime soon, as is increasing in mindshare. No arguments here, and there are plenty of REST bloggers out there popping champagne bottles over this statement.

The piece that I feel is getting lost in some of this discussion is that, as Anne points out in the APS blog:

REST is not the same as “plain old XML” (POX). POX refers to the format of a message payload, and it says nothing about architectural style. It just says that you don’t wrap an XML message with an XML envelope (e.g., SOAP or Atom). More to the point, not all POX applications are RESTful, and not all RESTful applications are POX.

This is probably the biggest myth right now. There are lots of people that are using XML over HTTP, thinking that they are using REST, when in fact, they aren’t. Again, Anne nails it with this statement:

REST is not simply technology–it’s an architectural style that’s fundamentally different from they way most developers design systems today. It requires a noun-oriented approach to designing systems rather than one based on verbs. I know quite a few people that have been studying REST for years who still struggle with RESTful design practices. Understanding the basics of the style is easy. Truly groking it and being able to apply it to real-world situations is much harder.

This is where I’d like to see some constructive conversations. When we’re talking about SOA, we’re talking about services. Services represent capabilities. Capabilities are typically associated with verbs. REST is resource-driven. To me, that would mean that I should apply it where I need a resource oriented architecture, not a service oriented architecture. The question is where is this appropriate? Clearly, there are many calls that do nothing more than retrieve data, and it would seem that a REST-based approach for this would work very well. The question, however, is whether a resource oriented view is sufficient.

One of the things I like about SOA is that it can help establish better lines of ownership, which theoretically, can allow IT to operate more efficiently. Because these are based on services, however, they are more likely to be aligned along functional boundaries rather than on resource boundaries. Resources are shared across functional boundaries, so if my unit of composition is at a resource level, how do I deal with these concerns? I’m not suggesting that it can’t be done, but I think this is where there is lots of room to grow, as Anne points out. In the SearchWebServices example, Anne provided a light bulb example, which (pun intended) should turn on the light bulb for people on what they’re getting into:

“A REST application to turn on and off the lights in your building will require you to design a URI for every light bulb and then you send it on/off messages,” she explained. “It’s not like I have a single service that manages all my light bulbs. It’s a very different approach to designing a system. And it’s going to be really hard for developers to get their hands around it.”

So what is my opinion on all of this? If it’s doesn’t come across from my blog, I tend to very pragmatic. I think there are places for both WS-* and REST, and that will continue for the foreseeable future. REST makes lots of sense to me when we’re dealing with browser-based clients (e.g. AJAX). WS-* makes lots of sense to me for service-to-service interactions. I do fall more on the side of formal interfaces, and as a result, I want to learn more about WADL. I’ve yet to see a solution that is doing REST by the book, most examples I’ve been fall more into the POX/HTTP category, or using an HTTP GET with query parameters to return data as XML (all read only). That doesn’t mean they don’t exist, I just haven’t seen them. In any case, debates such as these keep things interesting. There’s always risks that it will strictly be a battle of religions rooted in opinions (which never get resolved). Involvement of people like Anne Thomas Manes and others that fall into the pragmatic group in the middle can ensure that the debate progresses to appropriate application of these approaches to where we leverage the strengths of both, and minimize their use in areas where the weaknesses are exposed.

I was recently having a conversation regarding data services and my recent posts (here and here) on horizontal versus vertical thinking put it in a new light. My experience has shown me that data services are one of the most controversial topics within an enterprise.

Nearly all solutions, somewhere along the line, are going to need to access some form of structural data, typically from one or more relational databases. The debate around data services begins with a discussion on whether or not all access to the data should go through a service or not. I’m not going to get into that part of the debate in this post. The point of this discussion lies more with the organizational challenge of creating a data services layer.

One reason (but certainly not the only one) that is sometimes used to justify a data services layer is to maintain control over how the databases are used. All it takes is one bad SQL statement to cause a significant impact to all systems using that database. When organizations see this problem, the instinct is to centralize the development of data services. By creating a team with expertise in database access technologies, the assumption is that you’ll get high quality services that will prevent system failures due to data unavailability. The challenge with this approach is that this assigns ownership and service boundaries along a technical domain: data access. Per my previous postings, assigning ownership based on technical boundaries implies a horizontal domain. Horizontal domains are ones where the capabilities are becoming commodities and can be done in a standard fashion. Service teams in a horizontal domain typically strive to minimize the number of services involved. This is where the challenge occurs. Data access is anything but a commodity. While the techniques used to access data may be standardized (SQL), the structure of data and the combinations of data desired by consumers is certainly not.

In his book Human Interactions,
Keith Harrison-Broninski pointed out that everyone deals with information differently. We file it differently, we take notes differently. Trying to standardize information access is inherently problematic. Why is this? Because information retrieval operates in a vertical domain. We’re retrieving information for the purpose of doing a particular task. So, we have a services team that is viewing the world in a horizontal fashion, with their primary goal being to minimize the number of services they provide. On the other side of the equation, we have vertical thinking consumers that want data for a particular purpose in a format that is optimized for what they need to do. Clearly, if the consumers have the most influence, the goal of the service team to minimize the number of services is not going to happen. If the service team has the most influence, the goal of the consumers to have optimized services is not going to happen. We simply have a situation where the two (or more) teams involved have competing goals. You can’t say that either one is right or wrong, only that they are competing with each other. There can be sound justification on both sides, so the organization must mediate and try to create a win-win situation.

I think this subject is a great example of the concepts of horizontal and vertical thinking. Things can go very smoothly when both the consumer and provider are consistent with their understanding of a particular domain as either horizontal or vertical. When there is disagreement, and one group sees things as horizontal while another sees things as vertical, the relationship is going to be problematic. If you can make it a win-win situation, that is the best scenario. If one party or the other simply has a view that is inconsistent with the view of enterprise architecture, that’s a communications and education issue for EA. EA needs to set the policies for what constitutes good architecture, and if a particular functional domain is deemed to be horizontal (or vertical) that information should be well communicated so the individual solution architect approaches its usage properly.

I just listened to Elizabeth Book’s podcast with Anne Thomas Manes of Burton Group and Joe McKendrick discussing Service Averse Architecture. They definitely echo statements that I and others have been making for some time. Particular quotes that I want to highlight:

The companies that are good with SOA are likely to be … companies that have forward-thinking management and are good at everything else, as well … The organizations that really could use SOA, that need that flexibility and agility, are the ones not likely to be adopting SOA.
– Joe McKendrick

Joe and I both had posts on this topic back in October of 2006 (see here and here). This is simply the nature of way things progress. You have a certain class of organization that are the leaders that have a high percentage of success at whatever they do. The problem, however, comes with the middle class. What we’d like to see is that middle class of organizations merely trailing the leaders, but still eventually achieving success. Instead, we’re at risk of seeing the “haves” and the “have nots.” Anne calls out that Service Averse Architecture is the status quo, and that’s a shame. As long as it continues to be the status quo, we’re not going to progress as an industry, and if anything, it fuels the fire of those that question the relevance of IT. After all, if businesses continue to thrive despite having continued dissatisfaction with IT support, then is IT really a differentiator worth investing in, or is it simply a necessary cost center where the focus should be on driving costs down to rock bottom levels? I believe IT can be a differentiator, but like Anne, I do believe it’s a very select few who are able to do so.

The same type of thinking does hold true in other business support areas. Is HR simply a necessary evil, or can it be a differentiator? Look at Southwest Airlines hiring practices and the experience they are able to give their customers and you can see the impact. Despite having no reserved seats, their customer satisfaction (and profitability) is far better than most airlines. Thankfully, a few shining stars can go a long way in keeping efforts alive to ensure that we don’t have the “technology haves” and the “technology have nots.” It’s why I am supportive of efforts like the SOA Consortium.

The big issue is that most organization have a culture and an incentive system that absolutely discourages the adoption of good SOA practices. That is, everything is project focuses and every project is completely focused on delivery of that project as quickly as possible at the least possible cost and there is no interest whatsoever in figuring out, “well, what is it that I’m building that should be implemented as a service that other people can use?”
– Anne Thomas Manes

I was happy to hear Anne call out the problems with the project-based culture prevalent in IT organizations that I also commented on in April of 2007. There are huge cultural changes that need to be made, but most organizations are not doing so. It’s an extremely difficult problem, because you need to figure out an incremental way of doing it. Anne goes on to comment:

So much of SOA is tied up in things like Master DM and ID M and a reasonable DA and BPI efforts and all the EA things people are doing. You have to take this cross-organizational perspective and plan what you’re going to do and not just say, “Oh, I’ve got a new project I’m going to let my developers figure out what services to build.” It’s challenging because you have to think globally but you do actually have to implement on a local level. But you just can’t depend on the local people to figure out what they’re supposed to be doing. They need guidance from above.
– Anne Thomas Manes

This articulates the challenge well. You can’t simply say tell the developers to build the right services, even if you’re allowing them to challenge the scope boundaries of the projects, without having appropriate guidance from above. I’ve see a number of efforts where developers are honestly trying to do the right thing, but they don’t know the right next steps to take. The project-based culture is so ingrained that even when they are able to identify appropriate services that have broader value, they don’t know what to do to ensure that those services actually see broad use. Too often, it’s simply a “let’s build it and hope they come” approach, where the only guaranteed consumer is that initial project that kicked it all off. In short, not much has changed.

I talk about this as a lifestyle. If you want to become healthy and fit you need to exercise and eat right. Well, if you want your systems to be agile and flexible, then you have to adopt SOA fitness activities. You have to recognize what’s good behavior, what’s bad behavior, and try not to do the bad things, try to do the good things.
– Anne Thomas Manes

If you’ve been lucky enough to have someone really think about what it means to adopt SOA, you’ll understand exactly what Anne is saying here. SOA winds up touching all aspects of IT. If your doctor tells you that you are at risk of a heart attack, simply eating less fatty foods is not enough. There is no magic pill or single activity that will make your SOA adoption efforts successful. It is a lifestyle, and you need to be committed to everything that it may entail, rather than only doing it where it is convenient. We have to recognize that to make things better for the business, things have to change. If we’re only implementing minor changes, we can only expect minor successes. If your IT department is already very successful, minor changes may be fine. If your IT department is on a path to a heart attack, major changes may be in order.

Update: As usual, the timing of Dilbert is perfect. The Dilbert for Monday, May 22nd really captures the status quo and how there is far too much competition rather cooperation that occurs within the enterprise.

John Wu commented on my horizontal and vertical thinking post earlier in the week, and I felt it warranted a followup post. He stated:

Enterprise Architecture is a horizontal thinking while application development is a vertical thinking.

While I understand where John was coming from, I think that this approach is only appropriate at the very early stages of an EA practice. The first problem that an organization may face is that no one is thinking horizontally. This may go all the way down to the infrastructure level. Projects are always structured as a top-to-bottom vertical solution. While there may be individuals that are calling out some horizontal needs, unless the organization formally makes it someone’s responsibility to be thinking that way, it will have difficulty gaining traction.

Unfortunately, simply creating an EA organization that thinks horizontally and still letting the project efforts think vertically is not going to fix the problem. If anything, it will introduce tension in the organization, with developers claiming EA is an ivory tower and EA claiming that developers are a bunch of rogues that are doing whatever they want in the name of the project schedule.

If we characterize where the organization needs to go, it’s where both EA and the development organization are thinking both vertically and horizontally. This does come back to governance. Governance is about decision making principles to elicit the desired behavior. The governance policies should help an organization decide what things are horizontal, what things are vertical, and then assign people to work on those efforts within those architectural boundaries. Right now, many organizations are letting project definitions establish architectural boundaries rather than having architectural boundaries first, and then projects within those boundaries. Project boundaries are artificial, and end when the project ends. Architectural boundaries, while they may change over time, should not be tied to the lifecycle of a project.

So, EA should be concerned with both the vertical and the horizontal nature of IT solutions. Based upon the corporate objectives, they should know where it is appropriate to leverage existing, horizontal solutions and where it is appropriate to cede more control (although maintaining architectural consistency) to a vertical domain. Two systems that have some redundant capabilities but are architecturally consistent at least create the potential for a consolidation at some later point in time, when the corporate objectives and policies change. In order to do this, the project staff must also be aware of both the vertical and horizontal nature of IT solutions.