Todd Biske: Outside the Box

Archive for the ‘SOA’ Category

For those of you that are seeing “Governing the Garden” in your feed reader, but no post here, it’s coming. I accidentally posted it before I had completed it, then deleted it. For some strange reason, it’s still in the RSS feed. Quirk of WordPress, I guess.

Dana Gardner has posted the latest edition of his Briefings Direct SOA Insights podcast series, which is a discussion on virtualization and IT operations efficiency. Besides myself, we had a big group for this discussion including Jim Kobielus, Neil Macehiter, Dan Kusnetzky, Brad Shimmin, JP Morgenthal, and Tony Baer.

I find both of these topics very interesting and was glad to be part of the discussion. Operational management is an area ripe for improvement in many organizations, and it’s something that doesn’t get a lot of attention, because the impact to the business is largely ignored unless something breaks. Somehow, we need to move from a firefighting mode to a value-add mode where it’s more about the collection of data during normal operations to assist in the continued success of the business.

You can read a full transcript of the discussion here.

It’s about 6:20 in the morning, and I’m presently on a 5 hour bus ride with a bunch of IT staff headed toward some facilities of my employer to learn more about their business. Before I get into the topic for this entry, I certainly want to give kudos to my employer for setting up this opportunity for IT to learn more about the business. With the long bus ride, I’m trying to get caught up on some podcasts. I’m presently listening to a discussion on SOA Management with Jason Bloomberg of ZapThink and Dana Gardner of Interarbor Solutions.

In his intro, Dana Gardner used the oft-mentioned and maligned term, silo. For whatever reason, it suddenly occurred to me that we always work within silos. Organizational structures create silos. Projects create silos. Physical locations can create silos. So, perhaps the right discussion shouldn’t be how to eliminate silos, but rather, how to choose silos correctly, work appropriately within them, and know when to redefine them. I’ve commented a bit on how to organize things, specifically in my post on horizontal and vertical thinking. For this entry, I’d like to discuss the appropriate way to operate within a horizontal silo.

A horizontal silo is one where the services being provided from that silo have broad applicability. A very easy example most organizations should be familiar with is servers. While there are multiple products involved based on the type of processing (e.g. big number crunching versus simple transactional web forms), you’d be hard pressed to justify having every project select its own servers. This area isn’t without change, one only needs to look at the space of VMWare and its competitors to see this.

In many IT organizations, when a centralized group is established, the focus can often be on cost containment or reduction. For many horizontal domains, this makes a lot of sense, but there’s a risk associated with this. When a group focuses on cost reduction, this is frequently done at the expense of the customer. Cost reduction typically means more standardization and less customer choice. To an extreme, the service team can wind up getting too focused on their own internal processes and expenses and completely forget about the customer. This is neither good nor bad, only something that must be decided by the organization. I can do a lot of my household shopping at either Target or Walmart. I’m probably going to have a better experience at Target, however, it will probably cost more than Walmart. What’s most important to you?

Within the enterprise, I’ve seen this dilemma occur in areas where some specialized technical knowledge is needed, but where the services themselves ore not standardized/commoditized. The most frequent example is that of Data Services. Many organizations create a centralized data services group to retain oversight over all SQL written. The problem with this is that while we may have a team that can write great SQL, we may not have a team that really understands what information the business needs from those queries. The team may try to minimize the amount of services available rather than give their customers what they need. What’s the right answer?

When establishing a service team, you need to think about your engagement model. Are you going to provide an outsourcing model, or a consulting model? In an outsourcing model, the service is largely provided without customer input. Customers simply pick from a list of choices and the burden is completely on the service team to provide. Getting a server or a network connection may be much better suited for this category. In a consulting model, there’s a recognition that some amount of input from the customer is still needed to be successful. I can’t create a good data service without knowledge of the customer’s information needs. A consulting model is going to be more expensive than an outsourcing model. If an organization is judging the success of this team based on cost, however, that’s a problem. This is an issue with the success criteria, however. First and foremost, we want to be sure that the right solution gets built. When the services being provided are provided in a cookie-cutter approach, the emphasis can be on cost. When each service requires customization, the focus needs to shift a little bit more toward providing the right solution, with cost as a secondary concern. It may not be about creating reusable services, but usable services that are less likely to cause problems than a custom-built solution by staff without the proper expertise in a given area.

Who says blogging doesn’t pay? In my mail this weekend was a package from none other than Greg, of Greg the Architect fame containing a t-shirt (thankfully in adult sizes and not in action figure sizes) and a letter about his upcoming exploits. Greg is looking to leverage “Social Networking” (as he put it) in the development of future episodes and is looking for the help of the blogosphere. In his exploits to achieve SOA success, he’s dealt with vendors, trade shows, and industry analysts. What new topics would you like to see Greg take on? So far, REST hasn’t come into the mix, nor have we seen much exposure of the internal workings of Greg’s company (does it have a name?). I encourage you to go to Greg’s website and share you ideas. He was very explicit in his letter that credit will be given appropriately!

More and more, we’re seeing articles that are now questioning whether SOA is just the next overhyped thing that has come out of IT that has failed to deliver on its promises. Based on a conversation with colleagues at the SOA Consortium, I think at least part of the reason for this is that many organizations still don’t really understand what they’re getting into. At the same time, there’s been a debate on the Yahoo SOA group on SOA as a business thing versus an IT thing. All of this seems to point back to a lack of a clear definition and picture of what we’re talking about.

As I started forming my ideas to blog on this, I was listening to a Software Engineering Radio podcast, which just happened to be a discussion on SOA with Nico Josuttis, who has just authored a book called, “SOA in Practice: The Art of Distributed Systems Design”. In the discussion, I thought Nico put it very well when he stated that SOA is essentially distributed system design where the distributed components have distinct owners. He then said the essentially, there won’t be any projects where a single team has full end-to-end control. It was that second statement that is the key message.

There’s no shortage of organizations that are simply taking the same projects that they’ve been doing for years and simply producing services now as part of them. Is this SOA? It is possible that the project could have been of significant enough scope (more likely a program) to where it included the development of services along with multiple consumers of those services, potentially yielding immediate benefits. The problem with judging SOA from this standpoint, however, is that it is unlikely to scale. Not all projects have this broad of scope. Secondly, it didn’t adhere to the second statement by Nico. By being underneath a common project or program, that effort had complete control over the entire domain. Yes, that project probably had multiple teams involved, but it’s likely that ultimate authority rested with the project architect and project manager, and if they said to do something, it was going to be done that way. Funding decisions and other governance issues don’t even come into play, because it’s all controlled by the project umbrella. An easy litmus test for this is to ask what the organization will do when a new consumer outside of the project comes along and wants to use one of the services that was (or is being) built, but needs some modifications. If it occurs while the project is still in flight, the project manager is likely to raise their hand and say, “Sorry, scope creep, can’t do it.” If it occurs after the project is complete, the new project may have no idea who to even talk to, because the whole structure around that service was based upon the project, and now that the project is over, it’s now being ignored, save for bug fixes/production problems. This isn’t a good situation.

I’m a believer that if you are adopting SOA, you’re committing to a fundamental change in the way IT operates. Of course, this assumes that there are opportunities for improvement within IT. If your IT department is delivering the right things to the business at the right time and for the right cost, great, you’re in the top 5% of organizations and have probably already embraced the changes, whether due to SOA or not. If you’re in the remaining 95% of organizations, you need to give some thought to where you want to go with IT. Is the use of shared services a goal? What are the domains where you know you’ll get some sharing? Is it more about business agility and being able to change the way things are wired in a more efficient manner? How will you tackle the problem of competing priorities when two projects want to leverage a new service, but have competing timelines, or even worse, where one project has control over one consumer and the service, and the other consuming project is at its mercy? How does the organization need to be changed to support this model?

Rather than try to paint my own picture of what this future state could be, I’m going to point all my readers to Nick Malik’s series on Joe Freeflier, part 1, part 2, and part 3. This does a great job in doing this. While this may represent an extreme example, and one that could take 5-10 years to reach, it does a great job in introducing the concepts and ideas that you need to be thinking about as you go forward with SOA.

Pete Lacey had a very interesting post entitled, “What is SOA?” I encourage you to go to his blog and read the whole thing, but I wanted to call out a couple of nuggets here for some comments. First:

“But wait,” I hear my SOA-loving readers say. “SOA is not about exposing business logic on the network. That’s just a technology thing. SOA is about the business! CxOs and business units don’t care about technology, they will only pay for business solutions.” Which always makes me scratch my head. What exactly does IT ever do that’s not about the business? Do they not work for the same company as the other BUs?

This is very reminiscent of the recent discussion on the business case for SOA, which I argued should be a business case for services, not SOA. Pete hits the nail on the head with his comment, “What exactly does IT ever do that’s not about the business?” Of course, it’s probably better stated, “Shouldn’t everything IT does be about the business?” After all, there’s probably some things going on most large IT shops whose business value is debatable. Anyway, what’s really interesting about this is how many activities within IT don’t go far enough to be about the business. IT is just as guilty of putting up walls as the business partners are. Recently, I had a discussion regarding instrumentation of services and collection of usage metrics. I argued that anyone who is acting in the capacity of a service manager should be looking at those metrics on a regular basis. Think about this. How frequently do you run into someone within IT who has recently put a solution into production who is actively monitoring the usage of it to determine if it is a business success? More often than not, I see project teams that quickly dissolve, allocated off to other projects, and no one paying attention to run-time usage unless some red light goes off because something’s broken. This type of monitoring should not strictly be the domain of “the business”. As IT, you are PART of the business, and you should be proactive in your monitoring and understanding of how your solutions are supporting the business. If you throw it over the wall into production, then the business-IT relationship is going to look very similar.

Pete also states:

“Well, it’s not just about business alignment,” another group of SOA advocates claim. “It’s really about governance.” Again, I’m scratching my head here. Everything is about governance! Software development (network-oriented or not) is about governance: ‘You must use a version control system; you must write unit tests.’ Moving systems into production is about governance. Updating a routing table is about governance. Hiring a new employee is about governance. Buying a plane ticket is about governance.

“No, no.” They go on. “With SOA there’s new things to govern.” That’s true, there are. But really, is it that much different than any other governance process? Not really.

I certainly agree that the need for governance is not just limited to SOA. What I’ve found in my work is that long term SOA success requires a fundamental change in the way IT operates. It’s not limited to SOA, however. SOA is merely the trigger that is bringing attention to the flaws in the system. As IT grows, there will be a need for governance, period, whether it’s for SOA or not. This doesn’t mean that SOA isn’t important, but it also means that there’s a risk that SOA becomes the banner under which all things wrong with IT are destined to be cured, and that’s a dangerous path. I’ve done work where the “SOA deliverables” contained a whole bunch of stuff that arguably were not specific to SOA at all. At the same time, if that’s what it takes to bring some attention to some things that need to be fixed, I’m not going to argue. Just be cautious in how much falls under that banner, because you can inadvertently jeopardize your SOA efforts on things that have nothing to do with it.

While doing my thinking outside the box, I ran across Joe McKendrick’s post discussing whether SOA can be boxed. Joe provided commentary on a blog entry from Jack van Hoof who called out that the major ERP-vendors could potentially deliver “SOA in a box.” He points out:

SAP offers a service bus, service registry, events registry, canonical data management, business processes, services deployments (!), business monitoring, business process management, security… out-of-the-box. Yes, of course the implementation must be tuned and configured. But it’s all there, out-of-the-box.

There’s a certain amount of truth to this statement. It’s probably better stated as services in a box, because after all, it’s a box. We have no idea what the underlying architecture of SAP is. While it may be the case that SAP or any other large ERP system could constitute the bulk of your infrastructure, that doesn’t mean that you’re suddenly purchasing an SOA. If new business needs come along, it’s now up to your ERP system to provide those capabilities. If they’re new, then it’s the ERP vendor who must figure out how to quickly make the changes necessary, if they even choose to do so, since it would have to be something with broad customer applicability to make financial sense for them. It’s certainly possible to build custom code on top of the ERP system, but you’re always going to have that dependency. That’s not necessarily a bad thing, you just have to choose wisely on where to leverage the ERP system.

I’d like to pick on one comment Joe made in his commentary. He stated:

…the idea of buying all SOA from one vendor flies in the face of the ultimate meaning and purpose of SOA — the ability to pick and choose tools, applications and services from any and all vendors. SOA is supposed to mean the end of vendor lock-in.

I don’t agree with this opinion. While services can be used to create an abstraction layer from vendor products, I don’t think it needs to be a goal. The factors that influence whether an organization leverages one vendor, lots of vendors, or no vendors really doesn’t come into play at all. What SOA should do is assist you in making appropriate vendor decisions. Just as I commented some time ago that SOA should neither increase nor decrease outsourcing, but instead ensure increase the chance that outsourcing efforts are successful, the same holds true for choosing vendors. By breaking the problem domain down to a finer-grained level (services rather than applications), I can make better decisions on the vendor products I choose. If they don’t expose services for the capabilities I need, I’m going to look elsewhere. The only thing that could start leading toward better insulation from vendor lock-in will be more standards in the vertical domains. There’s plenty of standards out there, but there’s probably far more spaces that are not standardized.

So, what’s my advice? I don’t think you can buy “SOA in a box,” you can only buy “services in a box.” Your enterprise architects need to be the ones defining the architecture, and then leveraging the architecture to ensure that not only your home grown systems, but also your vendor systems, whether from one or many, adhere to it.

Neil Ward-Dutton of Macehiter Ward-Dutton had a post recently entitled “Rethinking IT projects? Think service, not product, focus.” He called out a frequent mantra of mine, which is that we need to get away from the typical project-based mentality and toward a product-based mentality. While Neil agrees that we need to get away from the project-based mentality, he questioned whether a product-based mentality is the right approach either. Instead, he advocates a service-based focus, where “service” in this case is more akin to its use in ITIL and IT Service Management (my interpretation, not his).

If you dig into his post, and if you’ve been following my own posts, I think you’ll find that we’re essentially saying the same thing, but perhaps using different terminology. Neil states:

The trouble is that taking too literal a view of IT delivery through the lens of product management can prevent you from reflecting reality the way that “customers” (regular business people in your organisation, and quite possibly those external customers that ultimately pay all the salaries) see it.

I couldn’t agree more. I’m a huge advocate for usability, user centered design, etc., so anything that involves assumptions on what the user needs rather than going out and actually involving the customer is definitely red flag in my book. Personally, I don’t even like to use the term customer when talking about IT delivering solutions to the business where the business is the end user. The business and IT should be partners in the effort, not a customer/supplier relationship.

Futher in the post, Neil makes the comment:

If you take too much of a product management centric view, the danger is that you focus all your energy creating the right kind of development and deployment capabilities, without thinking of the broader service experience that customers need and expect over the lifecycle of a long-term commitment. IT operations is where the rubber meets the road, and where customer expectations are met or dashed. Too simplistic a focus on product-style management for IT delivery perpetuates the development-operations divide and squanders a great opportunity.

Personally, I don’t view what Neil describes as good product management. If your definition of product management is simply a chained sequence of development activities, you’re missing the boat. Only through excellent operational management and customer interaction can you make appropriate decisions on what those subsequent activities should be. It’s not all about development. I think this point was made very well by Dr. Paul Brown in his book, “Succeeding with SOA: Realizing Business Value Through Total Architecture” which I was fortunate enough to review with Dana Gardner. He made very clear that projects are only successful when they deliver the promised business value. This value doesn’t come when the software is deployed into production. It comes after it. It may be one month, it may be one year. I’ve been involved with a number of projects that defined the project mission statement in terms of “delivering something by such-and-such date for $$$.” While meetings dates and budget are very important, they don’t define success. Delivering business value defines success, and the effort needs to continue to ensure that happens even if there are no development activities occurring. This brings in Neil’s notion of service, and it is absolutely a critical component to success.

In a SearchWebServices.com article, Rich Seeley asked the question, “Who owns the SOA business case?” I think this is asking the wrong question. The business case should not be for SOA, but for the services. While some may read this and simply think that I’m nitpicking on semantics, I think it’s an important distinction. If the business case is for SOA, what’s the scope of that SOA? Is it an application architecture? An architecture that encompasses a line of business? An architecture that encompasses the entire enterprise?

The business case should provide the justification for what gets built. The services are what will be built. The article used the term “SOA project” numerous times, which is a pet peeve of mine. I don’t think there should ever be a “SOA project” as you can’t build a SOA. So, the reality is that we have business projects, which probably had an owner assigned the same way that it’s always been done for all business projects.  As part of the project, one or more services are going to be constructed. This equates nicely with Miko Matsumura’s analogy, where he described the interaction between an electrician and a home owner. The electrician likely knows more about the electrical system than the home owner, but the home owner is the one picking the actual light fixtures. Location may be a joint decision, as the home owner may not understanding implications of particular locations. The decision on what goes on in the walls is probably exclusively the responsibility of the electrician (based on the local building code). There are always exceptions. My father was an electrician in the Chicago area, so he and I have a preference for wires flowing through pipe rather than the use of romex, and may choose to have more say in that decision. The same may hold true for a tech-savvy business owner.

Both Tony Baer and Dana Gardner have posted blog entries in response to the article and get closer to the notion of what the right thing to own is. Dana suggests that ownership should reside at the business process level. This is one way of slicing it, but it’s not the only way, and therefore, it’s probably not the right way. In my opinion, ownership comes into play at many levels. A service can be shared across multiple applications. A service can be shared across multiple business processes. If ownership of the shared service is assigned to a single application or business process owner, there’s risk. The right way is to draw a line between business ownership of a service and business ownership of service consumers. You’ll probably need both. Now the risk associated with this approach is the number of individual owners may grow quite large. While I do think that an organization should try to make some determination up front on the potential for a service to be shared, I also know that whatever direction they go, things will change. So, it is important to have an idea on how to change ownership of a service after the fact. Just as a commercial organization may begin with “bundling” and eventually converge multiple products into one, so may be the case with services. If this makes sense to you, then you’re grasping an aspect of the problem that hasn’t been discussed yet, which is, “What does a service owner actually do?” The owner of the business case, and ultimately, the service, is not simply the source of funding and scope decisions for the project that builds version 1.0, but must be the owner of the whole service lifecycle from version 1.0 to the decommissioning of the last version, handling the onboarding of new consumers, “service” to existing consumers, etc.

I’m back from my vacation and have a few topics queued up for some blog entries. The first one that I wanted to cover was the topic of events and SOA. The relationship between EDA, CEP, and SOA is one that pops up on a regular basis, however, in my opinion, it still hasn’t reached a sustained level of interest. There was a big peak some time ago when Oracle and Gartner used the ill-fated moniker SOA 2.0 to represent the combination of SOA and EDA, but once the backlash died down, the discussion around events faded back into the background again. Thankfully, it hasn’t gone away completely. One of the more recent publications on the topic came from Rich Seeley with SearchWebServices.com in this article. He stated that, “The relationship of complex event processing (CEP) to service-oriented architecture (SOA) remains, in a word, complex.” Why is the case?

The article included quotes from Jason Bloomberg of ZapThink comparing EDA to SOA. While I usually agree with the ZapThink guys, I disagree with Jason’s quote that there is no particular reason to distinguish SOA from EDA. Jason points out that all service messages are essentially software events which “contain all the information you’d ever want about the behavior and state of your business.” At a technical level, there’s nothing wrong with Jason’s statement. Where there are differences, however, is in the intent of the message. A message associated with an interaction between a service consumer and a service provider is either a request for the provider to do something or a response from the provider back to that consumer. The fundamental difference, in my opinion, is that these messages are directed at a specific destination. While you can certainly intercept these messages and use them for other purposes (and I’d argue that doing for so for business intelligence analysis reasons is a good thing), there’s a risk involved in doing this because now there are unintended side effects. In contrast, events have no requirement to be directed at any particular destination, typically using a publish-subscribe approach for distribution.

Let me get back to the question of complexity. I will admit that discussions like paragraph above are part of the reason that people find EDA and CEP hard to grasp. Often times, discussion in the blogosphere will focus on areas of disagreement, losing sight of the areas of agreement. I’d argue that if you talked to Jason and myself on the relationship between SOA and EDA, 80% of what you’d hear would be consistent. So, just because Rich Seeley received a number of different takes on SOA, CEP, and EDA, doesn’t mean it’s complex.The right question we should be tackling is how to make events, EDA, and CEP more interesting, building on the natural relationship to SOA. As I’ve previously stated in my uptake of CEP post, I don’t think that most organizations are ready for CEP and EDA yet. The debates that are occurring in the blogosphere and press are being made by people that have a vested interest (typically vendors, but you could argue that niche analyst firms do as well) in creating “buzz” about the topic. As a corporate practitioner, I have no such vested interest except where it makes business sense for my employer. So, I need to ask the question on how to make CEP and EDA relevant (and interesting) to the business. The challenge with it, and why it was important that I wrote about my difference of opinion with Jason of ZapThink, is that events, on their own, don’t do anything. A service performs some function. It does something. The business can grasp this. An event is just a nugget of information. A collection of events presented to business stakeholders are not going to be very meaningful until you start doing something with them. As a comparison, let’s look at baseball. If you watch or listen to a baseball game, you’ll get a barrage of statistics. Are they useful? Some managers, like Tony LaRussa of my hometown St. Louis Cardinals, have always made extensive use of the data. Has it made him more successful? We’ll probably never know. We can certainly say that he’s been a successful manager, but can we tie it specifically to his use of event capture and analysis? There are probably other managers or baseball pundits that would argue that the cost of collection and analysis isn’t worth it.

The same thing holds true for EDA and CEP. There is a cost associated with the generation of events. There is a cost associated with the analysis of events. What’s missing is the benefit. To get this, we need to do analysis of the business and come up with suitable justification. For a domain such as risk analytics associated with securities trading, the justification is there. Complex analysis of the trading and news events occurring in real time can result in better timed market activities with millions of dollars in potential benefits. In other domains, it may not be as crystal clear. If an organization has a stated goal of better knowledge of their customers, it would seem that event capture and analysis could assist in this, but how do we quantify the potential benefits? Just as with SOA, I think a key to this is selecting an appropriate proof-of-concept and then pilot. Some event capture and analysis can be done without purchasing any new infrastructure. There’s nothing wrong with performing analysis on a week’s worth of data that takes another week to complete if the end result is valuable, business relevant information. As Jason suggests, you can use service messages as your starting point for analysis, so if you’ve got audit logs of them, you only then need an analysis engine. Every organization already has many of these, and I’m not talking about a BI system. I’m talking about employees. While we may not capture all of the correlations, most of us are pretty good at spotting trends. It’s simply a matter of having someone look at the information that’s already there. Guess what that activity is? It’s business analysis. Do the analysis, understand the business, create the business case, and go make things better.

Two recent posts by Jeff Schneider and Nick Malik, recently brought some attention to a very important aspect of SOA, which is funding it.

Jeff’s post, entitled “Budgeting for SOA” gives a great breakdown of all aspects of adopting SOA, which based on my experience, would certainly be a multi-year effort. He begins with establishing the SOA Foundation, which includes establishing a strategy, roadmap, reference architecture, and governance strategy, continues on with SOA Infrastructure Realization, establishing a SOA Governance Team, performing architectural analysis, training the staff, and then building the services and their consumers. Jeff articulates the costs associated with each of these for a typical organization based upon his experience.

Nick’s post, address the other side of this, which is where to find the money. In his post, SOA Economic Model: Avoiding Chargeback with Transaction Ration Funding, Nick calls out his disdain for chargeback models, and instead presents an alternate view whereby shared resources are funded out of a fixed operating budget that is funded through some form of flat tax. The teams that are funded by this pool are then allocated funds based upon the amount of transactions they process. While Nick presents a very simple model based upon the number of transactions, I’m sure other funding models could be used since a simple request count doesn’t take into account the complexity of the services, but his model is accurate in that it properly incents the service development teams. The teams that use the services are going to pay the tax no matter what, so they’re also now incented to make use of the services being provided, unlike a chargeback model whereby they pass less if they don’t use the shared services.

I think both of these posts do an excellent job of laying out the possibilities. Unfortunately, I wish more organizations were at the state where they have to worry about these factors. It seems that the vast majority of organizations are simply trying to build services out of existing efforts. Unless those efforts are sufficiently broad in scope (and surprisingly, many organizations I’ve seen do have at least one major initiative that typically qualifies), it’s unlikely that services will be created that will have broader applicability. Now I consider myself an optimistic person, but I also recognize that this is not an easy challenge. To get to what I envision and what Jeff and Nick describe represent fundamental changes in the way software development takes place. To do this will mean leveraging staff that already operates out of a shared funding model, such as enterprise architecture, since their efforts merely need to be assigned by the EA manager. If the EA group establishes the appropriate strategic models, smaller projects with limited scope can now demonstrate how the will contribute to broader strategic goals, thus warranting funding of shared services, versus doing things they way they’ve always been done. All of this comes back to something that I frequently bring up in discussions around SOA adoption. If the only thing that’s changed within IT is that the same old projects that we’ve been doing for years now are creating services within them, do we really expect to see any kind of dramatic change within IT, or will it just be chalked up as another overhyped trend? I, as an EA, certainly plan on doing everything I can to make sure that’s not the case. We have to be careful not to bite off more than the organization can chew at any given time, but we also need to make sure that we are the impetus for change.

Tibco was kind enough to let me know that a new Greg the Architect video, Off the Grid, is now available. I just watched it and it is a riot. You can watch it here at the Greg the Architect site, or, if the plugin I just installed works correctly, you should see the YouTube player below this.[kml_flashembed movie=”http://www.youtube.com/v/CnhEfxxhg34″ width=”425″ height=”350″ wmode=”transparent” /] 

I had the opportunity to do a review of a book, and then discuss it in a podcast with the author and Dana Gardner. The book is entitled, “Succeeding with SOA: Realizing Business Value Through Total Architecture” and is written by Dr. Paul Brown of TIBCO Software.

You can view a transcript here, listen to the podcast here, or I’ve also added it as an enclosure on this entry.

I enjoyed this book quite a bit, and have to point out that it’s not your typical technology-focused SOA book. It presents many of the cultural and organization aspects behind SOA, and does a pretty good job. It tries to offer guidance that works within the typical project-based structures of many IT organizations. While I personally would like to see some of these project-based cultures broken down, this book offers practical advice that can be used today and eventually lead to the cultural changes necessary. Overall, I recommend the book. I found myself thinking, “Boy, if I were writing a book on SOA, these are things that I’d want to cover.” Give the podcast a listen, and check out the book if you’re interested.

Full disclosure: Outside of receiving a copy of the book to review, I did not receive any payment or other compensation for doing the review or participating in the podcast.

As reported by many bloggers/analysts/columnists, including this article by Tony Baer, BEA announced Project Genesis. Tony, in his article, stated that “there is considerable speculation that it will build on the [newly announced AquaLogic Enterprise] registry/repository.” I have a couple comments on the announcement that I wanted to discuss here.

First, we have yet another API in the metadata registry/repository space. Back in February, IBM created a whole bunch of brouhaha when they stated that they felt a new standard was needed for registry/repository communication, concluding that UDDI wouldn’t cut it. Now, we have the Metadata Interoperability Framework. Add this to UDDI, the WSRR protocol, HP/Mercury/Systinet Governance Interoperability Framework, and I’m sure this won’t be the end. You could certainly throw in LDAP for good measure, as well. Interestingly, no one has criticized BEA that I’ve seen for this move, but then again, they didn’t publicly come out and state that UDDI won’t cut it, either.

Second, given all of these APIs for metadata communication, it certainly does raise the question of whether metadata is really at the center of the SOA technology universe? Clearly, if all of the major platform vendors are now having to come up with custom protocols for communication with their registry/repository, it certainly means that there’s a lot going on with it. If you think about it, a metadata repository is a bridge between the design/development time world and the run-time world. Likewise, it also plays a key role in the run-time world and the world of operational analytics. Most people would also agree that most SOA efforts strive to separate non-functional concerns from functional concerns and allow for a policy-based approach to allow changes to be configured rather than coded.

We’ve certainly come a long way from the early days of the registry/repository where the focus was solely on design time discovery of services. Organizations that are solely focused on design time discovery of services still wrestle with whether they even need a registry/repository prior to reaching some critical mass of services. Vendors like BEA, IBM, and WebMethods are all certainly now pushing the Registry/Repository for much more than just this. I’ll admit, I have some concerns that we’re getting too far out ahead of the need of many enterprises. I’m a big proponent of a policy-driven infrastructure for SOA, but I’ll also admit that if you don’t have a sound understanding of your services and their consumers first, it’s not going to reap the benefits that are possible. Only time will tell whether we’re over-engineering based on what-if scenarios or if this level of sophistication driven by metadata is applicable to the masses.

One of my pet peeves when it comes to discussing services is when an organization gets into debates over whether a particular service is an “enterprise service” or not. These discussions always drive me nuts. My first question usually is what difference does it make? Will you suddenly change the way you construct something? It shouldn’t. More often than not, this conversation comes up when a project team wants to take a shortcut and avoid doing the full analysis that it will take to determine the expected number of consumers, appropriate scoping, etc. Instead, they want to focus exclusively on the project at hand and do only as much as necessary to satisfy that project’s needs. My advice is to always assume that a service is going to be used by the entire enterprise, and if time tells us that it’s only used by one consumer, that’s okay. Unfortunately, it seems that most organizations like to make the opposite assumption: assume that a service will only be used by the particular consumer in mind at that moment unless proven otherwise. This is far easier to swallow in the typical project-based culture of IT today, because odds are the service development team and the service consumer team are most likely the same group all working on the same project.

The natural argument against assuming that all services are “enterprise” services is that all of our services will be horribly over-engineered with a bunch of stuff thrown in because someone said, “What if?” The problem with over-engineering a service (or anything else) doesn’t stem from assuming that a service will have enterprise value, it stems from someone coming up with “what if” scenarios in place of analysis techniques to deeply understand the “right” capabilities that a service needs to provide. Analysis isn’t easy, and there’s no magic bullet that will ensure the right questions are asked to uncover this information, but I think many efforts today are not done to the best of our ability. As a result, people make design decisions based on a best guess, which can lead to either over or under-engineering.

I believe that if you are adopting SOA at an enterprise level, it will result in a fundamental change in the way IT operates and solutions are constructed. Requiring someone to prove that a service is an “enterprise” service before treating it as a service with appropriate processes and hygiene to manage the service lifecycle does nothing to promote this culture change, and in fact, is an inhibitor to that culture change. Will assuming that all services are enterprise services result in higher short term costs? Probably. Building something for use by a broader audience is more expensive, plenty of studies have shown that. On the other hand, assuming that all services are enterprise services will position you far better to achieve cost reduction in the long term as advocated by SOA.