Todd Biske: Outside the Box

Archive for the ‘SOA’ Category

I recently had a conversation with Ron Schmelzer of ZapThink and we started talking about how the nature of the entry point for enterprise users to interact with the information technology will change in the future. You’ll notice that I didn’t use the term “application” in that sentence and there’s a reason for that. Personally, I want to get rid of it. To me, it implies a monolith. It’s a collection of functionality that by its very nature goes against the notion of agility. When I look at a future state where we’re leveraging BPM, SOA, and Workflow technology, I see very small, lightweight entry points that are short and to the point. I’ve mentioned this before in connection with Vista Gadgets or MacOS X Dashboard Widgets.

Ron brought up a ZapFlash that came out over a year ago that he wrote called “SOA: Enabling the Long Tail of IT.” I didn’t make the connection at the time, but it makes perfect sense now. In the ZapFlash, Ron describes the “Long Tail” this way:

The Long Tail, a term first coined and popularized by Chris Anderson, refers to the economic phenomenon where products that are of interest to only small communities, and thus result in low demand and low sales volume, can collectively result in a large aggregate market. This large collection of small markets can significantly exceed the more traditional market that the most popular and high volume sales items can generate. For example, Amazon.com generates more business in aggregate from its millions of books that each only sell a few copies than they do from the top 100 best sellers that might each sell tens of thousands of units.
One quick way of summing up the Long Tail is by saying that thereís more opportunity in catering to a mass of niche markets than a niche of mass markets. Large enterprises in particular are composed of masses of such niches, operating in different geographies and business units, catering to specific demographics with tailored solutions to meet the needs of all constituents. And yet, the centralized IT organization that serves the needs of the entire organization is typically woefully unprepared to serve these masses of niches: large numbers of users with widely varying IT needs. How, then, can IT support the needs shared in common with all the business groups without overextending its centralized resource to meet the specific needs of each of the individual groups?

Fundamentally, we’re both talking about the same thing. What I describe as very lightweight user-facing entry points are the “long tail” of applications. They’re small, niche solutions that get the job done. Underlying all of this is a robust SOA that are the enablers of these solutions which is loosely-coupled from the user-facing needs. If you think about it, the long tail of application development today is the business user using Excel because they could get done what they needed quickly. I’ve even done this myself, and even progressed up to getting a simple database setup to do a bit more. We shouldn’t be on a quest to squash these out, but rather to figure out how to enable it in a manageable way. The problem is not that somebody’s Excel macro pulling data out of Oracle exists, the problem is that we’re not aware that it exists. Clearly, someone had a need to put it together, and if we can find a way to enable this to where we’re aware of it and our systems support it easily, even better. Personally, I think the technologies we have at our disposal today are on track for making this a reality.

I received two comments already on the Driving SOA post, one from Jason at ZapThink and one from Mike Kavis via his blog. Mike picked up that I suggested many of the responsibilities belong with the Chief Architect and correctly called out that this can be a lot of work to juggle. That being said, there’s absolutely no reason that the Chief Architect can’t delegate responsibilities to people on his or her team. Of course, if there is no architecture team, or if the architects are matrixed in from other organizations to where they have less-than-enterprise oversight, then this won’t work. This is where it gets back to a point I made at the end of my blog which asks whether the organization is biting off more than it can chew. This question needs to be asked before another body is brought in.

Jason specifically had some comments on the separation of responsibilities between the Chief Architect and the COO. He stated:

But a key point I’m making is that this person should be responsible for both the business process and architecture leadership. By separating the process responsibilities and assigning them to the COO, many organizations maintain the IT-centricity of their architecture efforts, which I see as a problem. That’s what I’m trying to address by combining the roles.

This is a good point, and if this is occurring in your organization, I think it indicates that the business/IT relationship is not where it needs to be. If the business and IT are simply all operating as “the business,” where they are seen as colleagues, peers, and partners, rather than IT being an order taker, I think the separation of responsibilities makes sense. If this partnership does not exist, an organizational change is one way of addressing it, but again, it is debatable (as we’re doing!) as to whether this approach will be successful or not. I’ve always said that organizations can both hinder or help large scale initiatives. What I’ve also found is that organizational changes take a long time to make up for a lack of trust in a company. The people have to be committed to working as partners, and if they aren’t, putting someone above them doesn’t always fix the problem unless there’s a big stick involved.

In the latest BriefingsDirect SOA Insights Edition, Dana Gardner, Jim Kobielus, Neil Macehiter, and Joe McKendrick discussed, among other things, Microsoft’s recent announcements. The conversation started very similar to some of my own comments on the subject with this sense of deja vu. Neil Macehiter made a great point, however, that shows that this isn’t simply a rehash of model-driven architecture. He stated:

…they are actually encompassing management into this modeling framework, and they’re planning to support some standards around things like the service modeling language (SML), which will allow the transition from development through to operations. So, this is actually about the model driven life cycle.

This reminded me of my trip to Redmond in 2005 for the Microsoft Technology Summit. At the summit, we were shown an internal tool, I think from the Patterns & Practices group, that presented a deployment model of a solution. I recall a number of us going, “we want that.” If Microsoft has taken steps to integrate these models into the development and run-time management tooling, this is an excellent step, and certainly something beyond the typical model-driven development of the BPM suites. At a minimum, these capabilities should be enough for people to at least track the ongoing progress of the Oslo effort.

The second thing that came up, which again was consistent with some recent blogs of mine (see Registries, Repositories, and Bears, oh my! and Is Metadata the center of the SOA technology universe?), was the discussion around the metadata repository at the heart of Microsoft’s strategy. Dana pointed out that “there really aren’t any standards for unifying modeling or repository for various models” with some comments from Neil that this is very ambitious. First, I’d have to say that Microsoft trumped IBM on this one. Remember when IBM announced WebSphere Registry Repository and stated that they’d be coming out with their own standards for communication with it? They were slammed by many analysts. Microsoft, rather than trying to operate in the narrow space of the SOA registry/repository, are talking about the importance of metadata in general. The breadth of models and associated metadata when talking about full IT product lifecycle (development and management), is far broader than what is typically discussed in the SOA space. AS a result, there are no standards that cover this completely, so the lack of standards-based integration is a non-issue, and Neil nails it by saying Microsoft is trying to get out in front of the metadata federation problem and drive others to comply with what they do.

Give the entire podcast a listen here, or read the transcript here.

Jason Bloomberg of ZapThink, in their latest ZapFlash, put a new spin on their old concept of the SOA champion and put forth a job description for VP of SOA. While he certainly suggested a good base salary for the position, I question whether a permanent, executive position around SOA makes sense?

If you look at the job responsibilities listed, the question I ask is not whether these tasks are needed, but rather, whether a dedicated person is needed for all of them. Let’s look at a few of them:

Provide executive-level management leadership to all architecture efforts across the enterprise. The directors of Business Architecture, Enterprise Architecture, Technical Architecture, Data Architecture, and Network Architecture will all be your direct reports.
Don’t we already have this? It sounds like a Chief Architect to me.

Drive all Business Process Management (BPM) initiatives enterprisewide. Coordinate with process specialists across all lines of business, and drive architectural approaches to business process.
Again, to me, this sounds like the responsibility of the COO.

Establish and drive a SOA governance board that incorporates the existing IT governance board.
This is the only one that I simply disagree with. If we’re speaking in terms of IT governance as defined in Peter Weill’s book, I think the IT Governance Board should be factoring SOA strategy and governance into their decisions. That is, IT Governance subsumes SOA governance, not vice versa. Of course, there are also aspects of SOA governance that are implemented at a much lower level within projects to ensure consistency of service definitions, etc. Once again, however, this should be handled by the existing technology governance processes. We’re merely adding some additional criteria for them to be applying to projects.

Establish and lead the SOA Center of Excellence across the enterprise to pull together and establish core architectural best practices for the entire organization. Develop and enforce a mandate for conformance to such best practices.
This gets into the technical governance I just mentioned. I certainly agree that a SOA COE can be a good thing, but you certainly don’t need a new position just to manage in it. Why wouldn’t the Chief Architect or a delegate lead the COE as part of their day to day responsibilities?

Manage a budget that is not tied to individual line-of-business projects, but is rather earmarked for cross-departmental SOA/BPM initiatives that drive business value for the enterprise as a whole.
The question here is whether or not the current organizational structure prevents cross-departmental initiatives from being funded and managed properly. It implies that the individual LOBs will be more concerned about their own needs, and not that of the broader enterprise. If the corporate governance principles and goals have stated that the use of more shared, cross-cutting tehcnologies are needed, then it’s really a governance problem. While an organizational change can assist, you still need to ensure that LOB managers are making decisions in line with the corporate goals, rather than that of their individual LOBs.

Work closely with the VP of Project Management to insure close cooperation between architecture and project management teams, and to improve project management policies.
Once again, why can’t this be done by the existing architectural leadership?

There were additional items, but my thoughts kept coming back to “shouldn’t someone already be doing this?” If the answer to this is “no,” then you must ask yourself whether or not you’re really committed to SOA adoption. If you feel that you are, but don’t have anyone assigned to these responsibilities, does the creation of a new position make sense? For example, if the organization struggles with getting LOB managers to produce cross-departmental solutions, will throwing one more person into the mix fix the problem, or just add another point of view?

As with many of the ZapThink ZapFlashes, there’s always a bit of controvery but lots of goodness. In this case, the articulation of the responsibilites associated with managing SOA adoption are excellent. Do you need a new position to do it? As with any organizational decision, it depends. If you are committed to SOA, but can’t make it happen simply because all of the possible candidates are simply to busy to take on these new responsibilities, then it might make sense (although you’ll probably need to add staff elsewhere, too, if people really are that busy). If you’re trying to use the position to resolve competiting priorities where there isn’t universal agreement on what the right thing to do for the enterprise is, a new position may not resolve it.

At the upcoming Gartner Application Architecture, Development and Integration Summit, I’ll be part of a panel discussion on funding SOA. I’ve previously posted on this in this entry, but I thought I’d bring it up again with the upcoming conference.

I’m very interested in hearing the experience of others on this topic. While there’s a lot of discussion about funding models, I still have yet to run into an organization that has had to actually implement one of these models. More often than not, I’ve seen one of two things:

1. A program of large enough scale where a number of services will be created with some in use by more than one consumer
2. Project-level SOA where a single project develops both the consumer and the service

There’s nothing wrong with either of these, but the thing to note is that these efforts did not require any change to the way funding of IT efforts occurs.

In discussing this with some colleagues, it seems that changes in how projects are funded really only comes about when reuse gets involved. In many ways, I feel that it is no different than dealing with shared (reused) infrastructure except that it is a bit more difficult to figure out how to partition the responsibilities.

A key question in all of this is how many services will be reused? If only 5% or 10% of services are reused, it is hard to justify changes to the funding model. But is this a chicken versus the egg scenario? Perhaps it is the way IT projects are defined to begin with which hampers the organization’s ability to identify services with reuse potential.

The point of this is that we’re still in the very early stages of SOA adoption. My sample base is very small, so I’m very interested in whether what I’m seeing is just an artifact of my small sample base, or if SOA funding is still a topic ahead of its time. This topic came up in a recent SOA Consortium call, and in order to reach out to a broader sampling base, I assisted in the development of a survey on the topic. It’s relatively high level and shouldn’t take very long to complete. I, and the other members of the SOA Consortium would certainly appreciate the input. It is intended for corporate SOA practitioners. You can access it here. Thanks!

It’s amazing how long it can take for some things to become a reality. Back in the corn fields of central Illinois in the early 90’s, I was in graduate school working for a professor that was researching visual programming languages. While the project was focused on building tools for visual programming languages, and not as much on visual programming itself, it certainly was interested. Here we are, almost 15 years later, and what’s the latest news from Microsoft? Visual programming languages. Okay, they’re calling it model driven development (which isn’t new either).

It will be very interesting to see if Microsoft can succeed in this endeavor. I suspect that they will, simply because Microsoft has a unique relationship with their development community, something that none of the Java players do. While there were competing tools for quite a few years, you’d be hard pressed to find an organization doing Microsoft development that isn’t using Visual Studio. You’d also be hard pressed to find an organization that isn’t leveraging the .NET framework. While the Java community has Eclipse, there’s still enough variation of the environment through the extensive plugin network that it’s a different beast. So, if Microsoft emphasizes model driven development in Visual Studio, it’s safe to say that a good number of developers will follow suit. As a point of comparison, BEA did the same thing over 5 years ago when they introduced WebLogic Workshop in February of 2002. This article stated:

BEA is calling WebLogic Workshop the first integrated application development environment with visual interfaces to Java and J2EE… “We’re radically changing the way people development applications,” said Alfred Chuang, president and CEO of BEA… Chuang said WebLogic Workshop could improve the application development and deployment cycle by as much as 100 times.

Hmmm… I’m getting a sense of deja vu. I’m hopeful that Microsoft can achieve better successes than BEA did. Point of fact, I don’t think it had anything to do with the quality of BEA’s offering. At the time, I had someone working for me look into Workshop. I was very surprised when the answer was not, “This is just a bunch of fluff, we need to keep writing the code” and instead was, “Wow, this really did improve my productivity.” Unfortunately, many developers will take their text-based editor to the grave with them so they can continue to write code.

In a similar vein, Phil Windley had a post on domain specific languages or DSLs. He points out that he is “a big believer in notation. Using the right notation to describe and think about a problem is a powerful tool.” Further on, he states, “GPLs (General Purpose Languages) are, by definition, general. They are designed to solve a wide host of problems outside the domain I care about. As a result, expressing something I can put in a line of code in a DSL takes a dozen in most GPLs.” Now Phil focuses on textual notations in his post, but I’d argue that there’s no reason that the notation can’t be symbolic. It may make the exercise of writing a parser more difficult, but then again, I’m pretty sure that the work I did back in grad school wound up having some in-memory representation of what was graphically shown in the editor that would have been pretty easy to parse. Of course, the example I used in my thesis wound up being music notation, which didn’t have such an internal representation, but I’m getting off the topic. If there are any grad students reading this blog, I think a parser for music notation is an interesting problem because it doesn’t follow the more procedural, flow chart like approach that we all too often see.

Anyway, back to the point. I 100% agree with Phil that a custom notation can make a smaller subset of programming tasks far more efficient. So much of what we do in corporate IT is just data in/data out type operations, and a language tuned for this, whether graphical or textual can help. The trend right now is toward graphical tools, toward model driven development. There will always be a need for GPLs, and we’ll continue to have them. But, given the smaller subset of typical corporate IT problems, it’s about time we start making things more efficient through DSLs, and model driven development. Let’s just hope it doesn’t take another 15 years.

InfoWorld published a collection of end-user stories on SOA this week, and the discussion on Leapfrog Enterprises caught my attention. In the article, Galen Gruman states that:

Leapfrog had many of the same goals that typify a typical SOA initiative: greater reuse of code, faster development time, and easier integration. But the company did not want to approach SOA as simply a changing of the guard for development tools and integration platforms. Instead it wanted to free its developers from conforming to a platform’s idea of best practices so they could focus on the applications’ functionality and use a wide range of development technologies as best for each job.

[Eugene] Ciurana [director of systems infrastructure for Leapfrog] did not want to force developers to all use the same transport. “The transport doesn’t matter,” he says. He chose to use the open source Mule ESB as a messaging backbone, relying on it to deal with transport interfaces. That way, “developers could focus as little as possible on the implementation of services,” he explains. Instead, their focus is on the functionality they are trying to achieve. The result is that developers tend to use HTTP as their transport mechanism, but some use REST (Representational State Transfer) and SOAP — “whatever works best or they’re most comfortable in.”

This caught my attention because it appears to be contrary to what I’ve seen at other organizations. Typically, the organization wants to constrain the platform to ease the integration problem and get away from the notion of “any-to-any” integration hubs. This may just be my misinterpretation, but it does raise an interesting question. How many constraints should be put in place? Interestingly, I’ve yet to run into an organization that has had to drive adoption of XML-based integration via enterprise architecture. The developers have slowly migrated away from whatever distributed object technology they were using and toward XML. The bigger challenge has been whether or not the XML contained the right information for broad consumption, not on whether or not XML was used. That being said, many EA teams are focused on the latter constraint (when to use XML or not). Knowing that there’s only so much that can be governed, what are the critical factors that EA teams should be making sure we get right as compared to the broad spectrum of things that we could be governing? Is it worth the pain to create policies regarding SOAP/HTTP versus XML/HTTP? The approach draws parallels to the big government/small government discussions that go on between the Democrats and the Republicans. The right answer is very dependent on the culture within IT, as I’ve stated often in this blog. Personally, I’m not a big fan of the integrate any-to-any approach. That being said, I also recognize that not everything is going to adhere to the constraints. I think it’s important to differentiate between your connectivity (mediation) infrastructure and your service enablement activities. Connectivity is about connecting two parties that adhere to the constraints for communication that have been adopted by the organization. Unless there’s only one way to communicate, there will be a need for mediation, for example, HTTP to JMS bridging. It’s important that the set of technologies be small, however. Service enablement is about taking something that doesn’t adhere to the standards and exposing it in a way that it does. We should strive to reduce the amount of service enablement over time, but the need for connectivity and mediation will always be there.

I was recently sent a copy of “SOA and WS-BPEL” by Yuri Vasiliev and Packt Publishing for review. The book is subtitled, “Composing Service-Oriented Solutions with PHP and ActiveBPEL.” After reading the book, the subtitle is much more accurate than the primary title. The book is first and foremost a guide for constructing Web Services using the SOAP extension for PHP and building BPEL processes using ActiveBPEL. Secondarily, there is a discussion behind the principles of SOA and WS-BPEL. Clearly, the right audience for this book is the developer community. If you’re removed from day to day coding, the book may not be as valuable.

If you’re looking for hands-on examples, this book has plenty of them. It includes all of the building blocks necessary from building your first service in PHP to creating an orchestrated process using BPEL. I felt that there was more emphasis on the coding efforts than necessary, however, and not enough on some of the theory behind it. This was evident in some of the early examples. In the sections on PHP, the examples result in a service that stores an XML representation of a purchase order in a database. The examples in the book take the incoming XML, create a PHP array representation of the XML, then convert it back to a DOM representation for storage in the database. While I do not know whether this approach was due to a limitation of the SOAP extensions, as an architect, it left me shaking my head. If the service is simply a pass-through to a database, there’s no reason to take all the time to parse the XML, bind it to some internal data structure, and then turn that internal data structure back into XML. Continuing on, the example then added XML schema validation to the mix, but it was performed by a stored procedure in the database. If you understand the role XML Schema validation has in security, odds are that XML schema validation will have occurred long before we even reach the back end database.

The section on WS-BPEL followed a similar vein. The bulk of the book was simply focused on walking you through the steps necessary to perform the actions in ActiveBPEL, rather than on building a sound understanding of WS-BPEL. There were pages upon pages of instructions on what menus to select, items to click, etc. I was very surprised at the lack of screenshots or graphical representations of the processes in the book. More often than not, they recommended using the Source tab in ActiveBPEL to compare the BPEL document in the book to what should have appeared after going through all the actions. My personal view on BPEL is that I don’t ever want to see it. I want to leverage the modeling capabilities of any BPM tool, and let the tool worry about BPEL behind the scenes.

Overall, for a book heavily focused on the developer community using PHP and ActiveBPEL (somewhat of a narrow audience, in my opinion), it’s certainly a good walkthrough to get your feet wet. It’s not going to give you the architectural skills you need, but it will move you through a lot of material very quickly. For people who are quick studies and just want some solid examples, you may find this a decent investment. For someone looking for more theory and architectural principles behind SOA and WS-BPEL, I’d probably look elsewhere.

Disclosure: This book was provided to me at no cost for the purposes of reviewing it. If you’re interested in having me review a book, please contact me at todd at biske dot com.

Okay, no bears, sorry. I read post from my good friend Jeff Schneider regarding SAP’s Enterprise Service Repository (ESR). He states:

At the core of the SAP SOA story is the Enterprise Service Repository (ESR). It is actually a combination of both registry and repository. The registry is a UDDI 3.0 implementation and has been tested to integrate with other registries such as Systinet. But the bulk of the work is in their repository. Unlike other commercial repositories, the first thing to notice is that SAP’s is pre-populated (full, not empty). It contains gobs of information on global data types, schemas, wsdl’s and similar artifacts relating to the SAP modules.

This now brings registry/repository into the mix of infrastructure products that SAP customers must make decisions regarding adoption and placement. Do they leverage what SAP provides, or do they go with more neutral products from a pure infrastructure provider such as BEA, HP, SOA Software, or SoftwareAG/WebMethods? The interesting thing with this particular space is that it’s not as simple as picking one. Jeff points out that the SAP ESR comes pre-populated with “gobs of information” on assets from the SAP modules. Choose something else, and this metadata goes away.

I hope that this may bring some much needed attention to the metadata integration/federation space. It’s not just a need to integrate across these competing products, but also a need to integrate with other metadata systems such as configuration management databases and development lifecycle solutions (Maven, Rational, Subversion, etc.). I called this Master Metadata Management in a previous post.

Back when Gartner was pushing the concept of the ESB heavily, I remember an opening keynote from Roy Schulte (I think) at a Web Services Summit in late 2005. He was emphasizing that an organization would have many ESBs that would need to interoperate. At this point, I don’t think that need is as critical as the need for our metadata systems to interoperate. You have to expect that as vendors of more vertical/business solutions start to expose their capabilities as services, they are likely to come with their own registry/repository containing their metadata, especially since there’s no standard way to just include this with a distribution and easily import it into a standalone RR. It would be great to see some pressure from the end-user community to start making some of this happen.

Both Collin Smith and Rob Eamon responded to my post regarding my participation in an upcoming panel discussion at the Gartner EA Summit asking for my thoughts on SOA, EA, and EA Frameworks, so I thought I’d oblige.

First off, I can be considered a “big SOA” advocate. That is, I think it needs to be applied at something larger than a project scope. I think it needs to be an initiative that is not tied to any one particular implementation project. This implies that it needs to be driven by a group in the organization that is not predominantly consumed with project activities. One obvious candidate, therefore, is an Enterprise Architecture organization. In fact, I can’t think of any other organization that is as good of a fit. Individual managers may embrace it, but they are typically not positioned to guide the organizational and cultural changes required, except at the highest levels. Their primary concern (and rightly so) is typically keeping their stakeholders happy. Enterprise-wide, or even department-wide adoption of SOA can be very disruptive in the short term. So, if I had to pick a group to drive SOA adoption, it will almost always be enterprise architecture.

As for whether “SOA folds into EA,” I agree with Rob’s comments. SOA doesn’t replace Enterprise Architecture, it’s simply one view of the enterprise. Today, one could argue that most organizations view the IT landscape as a collection of applications. Efforts like application rationalization or application portfolio management reinforce this notion. So, you could also say that today we have application oriented architectures. The unit of composition is the application. This isn’t flexible enough, as it is too coarsely defined. If we break these applications into smaller uniits, we now get to service oriented architecture, which I feel is a better way of describing things. Is it the only way? Certainly not. There may be value in a process-oriented view. We still need deployment-centric views that simply show physical, and now virtual, servers. We may need a network-centric view. These are all tools in the toolbox of the Enterprise Architecture team, and depending on your specific responsibilities within that team, some may be more important than others. As I’ve mentioned before, I have a background in human-computer interaction going all the way back to my college days, and one thing that I’ve always believed is that it is very unlikely that one view, whether it be a diagram, a user interface, will meet the needs of everyone. This is why I’m also not a huge fan of EA Frameworks. I think EA frameworks can be of great value when you’re starting out. The scope of EA can be daunting, and if you’re tasked with establishing an EA practice in an organization, it never hurts to begin with an established framework. When those frameworks become too focused on trying to make everything fit into a one-sized fits all approach, rather than on actually making the effort successful is where things can become problematic. Within EA, I don’t think it’s necessarily the fault of the frameworks, but more due to EA being an immature practice. While the concepts have been around for more than a decade, there are still many large organizations (at least in the area where I live) that don’t have an EA practice at all, or have only been doing it for 2 or 3 years. While my sample base is relatively small, my experiences have been that every organization does it differently. Some EA groups have significant authority, some have virtually no authority. Some groups spend all of their time engaged on projects, some have no engagement with projects. Some are committees, some are standing organizations. Some are exclusively focused on managed the technology footprint, some are actively involved with business strategy and business architecture. With this much variation, it’s hard for any framework to achieve wide adoption, because they’re simply not a good fit for the short term needs that the EA team needs to accomplish. When the primary artifact of EA tends to be intellectual capital (i.e. thought leadership, future state models, etc.), you need to have flexibility in how that capital is represented, because consumption is the number one factor, not standardization.

I’ll be part of two panel discussions at the upcoming Gartner Application Architecture, Development and Integration and Enterprise Architecture Summits. These are being held at the Rio Casino and Conference Center in Las Vegas the week of Dec. 3-7. In the App Arch summit, I’ll be part of a Power Breakfast discussing funding SOA on Tuesday morning at 7:30 am. In the EA summit, I’ll be part of a panel discussion jointly moderated by Gartner and The SOA Consortium discussing the relationship between EA and SOA on Wednesday at 3:30 pm.  I’ll be at the two summits from beginning to end (Monday – Friday), so feel free to find me and say hi.  One of the more enjoyable parts of these conferences for me is the networking opportunities.

Dan Foody of Progress Software had an interesting blog recently called UDDI in a Web 2.0 World. In it, he asks:

SOA What? With all of this Web 2.0 development, it’s clear that internet scale folksonomies work far better than taxonomies. On the other hand enterprises are, for the most part, stuck with UDDI-related SOA governance tools and their strict taxonomy and categorization mechanisms. The open question though… is this really a problem?

Aside: I love the use of SOA What? That’s exactly why I try to always say S-O-A. On the subject, however, I think Dan raises an interesting question. One of the questions I’ve asked some of the registry/repository vendors is “Can you be indexed by a Google Appliance?” Admittedly, I’m not a huge fan of taxonomy-based searching. At the same time, however, a typical enterprise asset repository may not have enough critical mass to get appropriate metadata for folksonomy based searching. The Web is filled with hyperlinks. How many links to a service detail page am I going to have inside a typical enterprise?

Personally, I’d rather try to find a way to build up the metadata than go crazy building taxonomies to support direct navigation. First off, you can quickly get into taxonomy hell where there are so many variations that you try to support that it becomes difficult to present to the user. Second, people are so used to using Google, Desktop Search, Spotlight, etc. Universal search is going to be a standard part of the office toolset, and we need to find a way to ensure relevant results get returned. This will likely require analysis of software development artifacts (including source code) and building up those relationships based upon presence within project repositories and the role of the user performing the search. A developer performing a search will want to see very different results when searching on “Customer service” than a business manager.

The challenge we face is that the documents and their metadata are scattered all over the place. I previously asked if metadata should be the center of the SOA universe. Neil Ward-Dutton replied that it the center of the universe, and is inherently federated. We need intelligent crawlers that can infer the appropriate relationships and feed this into the universal search engine. Is anyone out there leveraging a Google appliance or other universal search option to facilitate searching for services and other IT assets? If so, like Dan, I’d love to hear about your experiences.

Mike “The Mad Greek” Kavis had an interesting post on SOA Lessons Learned over at IT Toolbox. First off, a big thank you to Mike for sharing some experiences about an effort that didn’t go the way it was originally planned. We can learn as much from these as we can from success stories.

The part of the post that caught my attention began with this line:

As we started the second wave of projects, I mandated that all code should be delivered with test harnesses, the build process should be 100% automated, and testing automation should be part of the project deliverables.

Mike went on to discuss how automation and governance were quickly forgotten when the schedule began to slip. The surprising thing to me was not that these aspects were dropped when the schedule began to slip, but that the implementation of things like continuous integration and testing automation were tied to the SOA effort to begin with.

To me, this was indicative of something that I’ve seen at a number of places which is to use “SOA” as the umbrella to fix all things in need of improvement within the software development process. Continuous integration is a great thing and everyone should be practicing it. But if you organization hasn’t adopted it or created a standard, repeatable way of doing it, don’t target your pilot for another key initiative (like SOA) to try to make it happen. SOA adoption is not dependent on having a continuous integration system. If you have it, will it make SOA easier? Yes, probably, but more so because it’s making all development easier, regardless of whether you’re practicing SOA or not. Give it its own pilot where it can be successful in a very managed fashion, and roll it out to the rest of the enterprise.

Part of the problem is finding ways to adopt these improvements in the development process. Is the business going to care about continuous integration? You can argue that they should, but it’s really about internal IT processes. All too often, IT is left to take the congressional lawmaker approach and find some big project that will be sure to be funded and then push everything but the kitchen sink into it under the radar. This creates additional risk and often results in the project team biting off more than they can chew. It’s unfortunate that IT frequently has little ability to improve on its internal processes due to the project-centric nature of its work. Take another support organization like HR. While I’m sure some work in HR work is project-driven, a lot of the work is day-to-day operations. My suspicion is that organizations that are focused more on fixed-cost day-to-day work like this probably have more ability to take on internal improvement initiatives.

My point of all this is that an organization has to be careful on what they try to take on. There’s always opportunities for improvement, and the point should be quality, not quantity. Trying to take on everything is unlikely to lead to success on any of it. Taking on a smaller set of goals and ensuring that you do them well is a safer approach.

I just read about three blog entries that all had to do with Joe McKendrick’s recent post titled, “Enterprise SOA concept falls out of favor.” Others commenting on this post include Jeff Schneider and this one (author unknown).Personally, I’m tired of these articles that are lamenting SOA. If you read between the lines, the real story is that there are very few case studies of “big SOA” in the way it was originally hyped. Therefore, we need to start hyping “small SOA” to keep it relevant. Neither one of these is the right thing. If “Big SOA” isn’t succeeding, we should be asking why? Odds are, it’s because the organizations involved are simply trying to do IT the same way that they’ve always been doing IT, and merely hoping that by creating some “services” things will suddenly be better. Sorry, won’t happen. In this scenario, what’s really happening is “Small SOA.” Because we haven’t changed anything about the way IT defines its activities, the only hope of achieving any of the hyped goals of SOA is to find some existing projects that are of sufficient enough scope to create that potential. If they don’t exist, now it’s a pure bottom up approach where people are throwing arbitrary services out there and hoping for the best. Certainly having some services is better than no services, but you’d be hard pressed to claim that the organization has adopted SOA and shown value that a CIO would appreciate.Coming back to “Big SOA”, it also doesn’t do any good to say, “it must be enterprise” and try to get funding for some all-encompassing, top-down enterprise initiative. That’s unlikely to happen. What needs to happen is “right SOA.” In order to do it, however, you need to have some knowledge of where you’re going to get the most bang for your buck. I point back to my post on horizontal versus vertical thinking for this. If you don’t have an idea on how to break down your functional capabilities into this, you need to take the time to better understand your business. Do the analysis, and then make the decisions on where to apply it. If you only focus on projects at hand, you may have success on a project with appropriate scope, but then it all falls apart when you don’t have the next target area. This doesn’t have to be some huge, all-encompassing analysis effort, but it does need to be enough to ensure that the techniques you’re going to apply are relevant, will add value, and will be successful. If we don’t have this knowledge, how can we have any confidence that any new technology approach, whether it is SOA, REST, BPM, EDA, AJAX, Web 2.0, etc., will be successful?

I’m a frequent listener of Biotech Nation with Dr. Moira Gunn, available through IT Conversations, and have begun learning more about the world of biotechnology. In doing so, I realized that there are some good analogies between the practices in agriculture and what we need for governance from IT. I’m not the first one to use an agriculture analogy, as Neil Macehiter and Neil Ward-Dutton described the IT technology landscape as a garden in their book, “The Technology Garden: Cultivating Sustainable IT-Business Alignment.”

When we grow something, it starts with the seed. The seed needs to germinate to grow into something, be it corn, wheat, soybean, etc. There’s plenty of things that can go wrong at this stage, so seeds are normally pre-treated with some form of protection, such as an pesticide. Once the seed germinates and the plant begins to grow, the farmer or gardener now needs to worry about weeds and insects. In the past, this involved heavy doses of chemicals, be it pesticides or herbicides. The problem with these is that these chemicals tend to work selectively on particular bugs or particular weeds. If you had a different bug or a different weed, you threw more chemicals on it. Not only is there risk that these chemicals stay on the food that the plants produce, but also that the runoff does harm to the environment. Today, there are non-specific herbicides that will kill just about anything that is green and leafy. The problem with this is that is would normally kill the crop. To that end, biotechnology has allowed crops to be grown that have a resistance to these non-specific herbicides. The net result is that less chemicals are necessary. Rather than continual spraying of selective herbicides, a single spray of a non-specific herbicide can be used.

So how does this relate to governance? The seed is the typical IT project. This is the thing that we’re trying to grow. Pick the wrong seed and you have problems. If you don’t pre-treat the seed, it may never germinate. The theme here is that projects need to be positioned for success from the beginning. There are activities that take place at the inception (or even before) of a project that can make or break your efforts. Given that most IT shops are still very tactical in their activities, SOA adoption is still predominantly a bottom-up effort. If you don’t properly scope your projects, as well as watch it carefully at the beginning so that any service development efforts are properly scoped, your chances of SOA success (or long-lasting IT success for that matter), will be less.

Step two in the growing process was the use of biotechnology. These plants have genetics that create a resistance toward the bad things that could come along. In the IT project sense, this is all about making the right thing the path of least resistance. If you’re successful with doing this through education, tooling, and mentoring, you won’t need to worry about review committees and rigid processes, because your seed will naturally do the right thing.

Finally, there is still a need for some general oversight. Weeds can still grow in your garden, and those weeds can consume vital resources that your plants need to live. In the IT sense, this isn’t about the specific decisions on the projects, it’s about the resources allocated to the project. Even if your architecture is sound, if your resources aren’t empowered to do the right thing, or if they’re pulled in many directions by multiple projects, you’re going to have problems.

So, while many people have frequently used the city planning analogy for architecture and governance, I’m beginning to like the gardening/farming analogy even more. Create an environment that positions things for success, make the “right thing” the path of least resistance for the project, and finally, keep the weeds out, allowing the resources allocated to focus on making the project successful.