Todd Biske: Outside the Box

Joe McKendrick of ZDNet picked up on my so, uh, duh… blog entry regarding the new SOA 2.0 moniker. My previous post merely pointed to the petition to stop the madness. Something that is unfortunate in all of this is the actual reasoning behind the new moniker. Oracle and Gartner are trying to encourage the incorporation of events into SOA efforts. This, outside of the name, is actually a good thing. Bloggers like Brenda Michelson (eBizQ, elemental links) and Mike Herrick have posted on the importance of event driven architecture and its relation to SOA, as have I. Others out there clearly feel that EDA is a subset of SOA, and therefore there is no need to call it out. Personally, I think there is a risk that the need for events can get lost. A service is something that is available for the public good. It has to be requested. An event does not necessarily represent a request. It is merely a statement of something that has happened. What makes events so great is that anyone who is interested can choose to take action. The source of the event is not required to have any expectations of actions that will take place. It certainly can, but it doesn’t have to, unlike a service request. If we come back to the goals of SOA, a key one is agility. I don’t see how we can be agile without events. They bring exposure to happenings to a wider audience than the silo in which the action happened. That’s what it is all about. If we don’t break down the silos, we can’t be successful. Stop the madness. Ensure your architecture fully embraces services AND events. Just don’t call it SOA 2.0.

Brenda Michelson has posted a summary of current conversations between myself, Anil John, and Mark Griffin. My apologies for not posting this directly as a comment, but the response was getting long. She had four questions to continue the conversation that I’d like address as much as I’m able.

1. …understanding the business is critical in preparing an organization for SOA. But what else? As Mark asks, how can he ensure his organization is ready to deliver?

   My first comment refered to the importance of understanding what it takes to be an internal service provider. You can’t just build a service and forget about it. Outside of that, however, you need to have an understanding of the entire picture of SOA adoption. Ben Moreland of the Hartford gave a great presentation on their SOA Maturity Model. It’s a multi-dimensional effort. My dimensions would be (and they are very similar to what Ben presented):

   • Strategy: Bottom up, top down, etc.
   • Standards: What industry standards are you using? What internal standards are you creating?
   • Governance: How are you enforcing the standards? What oversight do you have in place (more on this later)
   • Capability: What is are the skills and capabilities of your staff? How are you training them?
   • Technology Infrastructure: Tools and technology for both development time and run time
   • Processes: What processes have you put in place to support all aspects of the adoption?
   • Architecture: What services have been created? What services do you plan on creating? Do you have references architectures, blueprints, and patterns to facilitate development?
   • Organization: How is the organization changing to support the effort? Do you have dedicated development teams?
2. And the other fork from Mark: “How do I get my IT organization more involved with the business to really understand it?”

   I recommend trying some user-centered design techniques. It’s been a long time since I’ve been able to practice them myself, but if you can find a way for your IT staff to do some field observation, they could learn a lot. Perhaps we need the business to sponsor a “Take your IT buddy to work” day. While I may be poking a little bit of fun, you’d be surprised what you could learn by just tagging along with a business worker for a day. Brenda, is there anything you can share from your own experiences or that of the Patricia Seybold Group?

3. What tools and/or methods are effective for top-down, bottom-up business and service modeling?

   I haven’t had the opportunity to dig into this personally. Clearly, existing modeling tools for software development projects work for the bottom-up case. For top-down, it’s all about business process modeling. I’ll leave this one for other people to offer suggestions.

4. How about that “elusive” governance? How do you sell Governance to IT and Business Leadership? Where do you get the money for people, process and tools? How do you convince people “Governance is good for you” rather than “Governance is a roadblock”?

   This is a tricky one. If your organization has some form of formal reviews, you’re already performing some governance, so it’s simply a matter of utilizing the existing gates to address service issues, such as reviewing the existing service library for possible services, determining where organizational ownership should lie, etc. If you don’t, then it’s time to start gathering some evidence to show the need. I would expect that sooner or later, a situation will arise that formal governance should have caught. The other way I would try to recommend governance is to first get some understanding on barriers to SOA, the biggest of which are the constraints placed by projects. If the project team stays constrained, odds are the effort will only yield results that are of value within those constraints. The only way to get around this is to give someone the responsibility to look outside of those constraints. Assigning the project staff to do this puts the project at risk. A better approach is to assign someone from outside the project to do this at the appropriate times. If you’re able to get people to understand that thinking outside of the box is a requirement for SOA, they may allow a pilot governance program to begin.

In a followup comment to the question on preparing for SOA, Mark Griffin asks whether it is possible to focus too much on the business, ignoring the IT deliverables. I think this is certainly a risk. While the failures were probably not due to an ignorance of technology, the business process re-engineering movement, noted by techniques such as Six Sigma, certainly left a lot of failed initiatives in its wake (at least according to books I’ve read, I have not personally been involved in such an effort). Likewise, I also believe that is possible to perform business process analysis and embrace business process technologies and completely miss all the benefits of SOA. As Anil John points out in his response, “Approaching SOA from both the Top-Down/Business Process perspective as well as the Bottom-Up/Service Factoring perspective allows for the identification of the re-usable aspects in a business process and to realize that re-usability using the service implementation technology of your choice.” This is well stated. The way to ensure that business process paralysis doesn’t happen is to embrace both a top-down approach and a bottom-up approach to your SOA and BPM adoption efforts. Strive to make the efforts meet in the middle. There is a challenge, still. Bottom-up efforts are more easily embraced because the starting point is well defined. Top-down efforts, are not so easily embraced. How high up do you go? I think this depends on your resources and the ability of those with business knowledge to contribute. You need to look broader than the bottom-up project at hand, but beginning at enterprise may be too far away to be practical. If the answer was easy, we’d all have done it by now.

Brenda Michelson, in her eBizQ blog, is encouraging a collaborative discussion around Business Driven Architecture. The first question posed by/to the blogosphere was on how to prepare your IT organization for Service Orientation.

MarkG’s stated that his “gut feeling says that IT shops that have a well established EAI practice or maybe have successfully embraced OO design will have the right mixed[sic] of governance and discipline to have a run at SOA.” On this, I have to disagree. I think a strong background in these technologies will assist in building service-oriented applications, but I don’t think it will go very far in yielding service-oriented architecture. The reason for this is that as long as all of the IT effort remains within the constraints of a single project, the organization will never understand it means to be a service provider. When a service provider and a service consumer are both working within a single project, the importance of these roles can be diminished, because there is close collaboration. Furthermore, the roles probably disappear when the project ends. What happens what the next consumer comes along? What happens when a project needs a service from a group that isn’t actively engaged on the project? How does the organization manage a service whose business logic spans multiple groups within the organization?

To me, the ability for an organization to succeed is going to be highly dependent on their ability to communicate, collaborate, and manage the service lifecycle, independent of whatever technologies are involved. I had the opportunity to comment about this in a recent ZapThink podcast. Please give it a listen and share your thoughts.

Thanks to Burton Group for putting on a very good conference in San Francisco this week, and more kudos to Anne Thomas Manes for putting together a great day of talks on SOA on Thursday.

This conference really demonstrated how building a network can make these events far more useful. Besides this blog, I participate in a Yahoo group on SOA as well as some less active user communities. It never fails that a few fellow participants show up at these conferences and the hallway conversations are quite valuable. At Catalyst, I had the opportunity to meet with Anil John from Johns Hopkins Applied Physics Lab, Shawn Furgason from Rockwell Collins, Ben Moreland from The Hartford, and Jeff Barr from Amazon, as well as some friends from the vendor community including Andrew Nash, Dave Hollander, and Miko Matsumura. I also met Brent Carlson from Logic Library for the first time.

All of you conference attendees that are practitioners, if your company policies allow it, I encourage you to get active and participate in the community! You can make your conference experiences even better by doing so.

James McGovern, as he so frequently does, encouraged a conversation on the evil enterprise architect. He makes a number of great points in this post.

1. “Many enterprises are getting it twisted by mistaking process for architecture…”
   First, since this is presented without the context, the message is that in many enterprises, the architect has fallen into a role of the gatekeeper. Rather than spending time on strategy, layers of abstraction, and modeling, they are consumed by tactical decisions for projects at hand. If you think about this, it really is a problem. How often does a building architect have to approve the detailed design and implementation decisions by the general contractor? Odds are, the architect is off designing the next building. Architecture establishes constraints, and it is the beginning of the design process. The problem is that is in IT solution development, these constraints exist too much in the architect’s head and not enough in something that is consumable by the general contractor on the project. As a result, the architect has to be more involved in the day to day project operations, and their role becomes more about the governance process than about strategic modeling. One additional item of note: processes and architecture do go together when it is about process modeling.
2. “What would happen if EAs started to think about creating new business value by modeling and ultimately simplifying the complexities of the business domain?”
   Life would be better! A barrier preventing this is that the role of Enterprise Architect is usually an IT role, not a business role. I wish it was the case that this wasn’t a barrier, as the efforts of everyone should be about the business. It shouldn’t matter what organization it comes from. There’s no shortage of articles written on how IT workers need to become more business aware. Likewise, there are some very tech-savvy business workers out there. In the perfect world, the Enterprise Business Architect could come from either the business or IT, and would have the full support of the enterprise in trying to improve the business.
3. “Maybe if us EAs started noodling the notion of enterprise service models that would be a good first step.”
   Noodle noodle. I am a proponent for an enterprise service blueprint, and I think we’re speaking to the same thing here.
4. “Business value can also be created by understanding the marketplace (bazaar) and even the social domain in which one plays in (aka community). What if we were to enable better ways for communities to communicate and interact with each other in a meaningful way? Would this enable new business opportunities?”
   I feel even more strongly about this. Unless we understand the cultures involved and the appropriate way to communicate and collaborate, we’ll never live up to our full potential. Sometimes we communicate poorly, but even more often, there is no communication at all. There are diamonds in the rough to be found, and only by continuing to improve how we communicate and collaborate, looking for new ways of getting things done can we continue to innovate and be more productive. A strange but accurate example of this is American Idol. The recording industry has been around a long, long time with set processes for finding new talent. There are many talented singers out there who simply have no exposure to that process. Along comes American Idol that challenges the way that singers are matched up with the recording industry, and a phenomenon is born. We must continue to search for ways to allow new ideas to get to the ears of people who need to hear them.

Ismael Ghalimi, on his IT|Redux blog recently posted that BPM 2.0 is Middle-Out. This was in response to another blog from Christopher Koch at CIO Blogs.

In Christopher’s article, he discussed that the business-side proponents from the historic business process re-engineering days now have technology on their side, BPM. Meanwhile, IT has SOA on their side. He concluded with a comment that BPM represented a top-down approach, while SOA represented a bottom-up approach.

Ismael picked up on that last comment to stress that BPM should not be a top-down approach, but rather, a middle-out approach where the process analyst is able to begin at the meeting point of IT and the business and work outword in both directions, one toward the business, the other toward the technology. He goes on to state that SOA and BPM are two sides of the same coin.

First off, it’s unfortunate that any controversy around SOA and BPM even exists, and I hope the analysts and columnists out there don’t try to fan the flames. Most SOA pundits I know completely understand the relationship between BPM and SOA, and how they should be inseparable. Any attempt to break them apart is a chicken and the egg discussion, and not worth the effort. Simply embrace both, and be done with it.

As for the whole top-down versus bottom-up discussion, here’s another case where the terms are used in so many different ways that it’s just making things confusing. Top-down versus bottom-up traditionally has to do with one thing: scope. Just because something may be driven out of the lines of business instead of IT doesn’t imply that it is top-down, and the same holds true in reverse. The middle-out discussion from Ismael has nothing to do with scope, either, it has to do with the business and IT operating as a team for a collaborative solution. David Linthicum discusses outside-in SOA with the increased number of externally hosted services available. Again, this has nothing to do with scope, it has to do with the physical location of the service processing. While Dave has good things to say, this does create confusion when an organization is simply trying to scope their effort.

If we agree that the fundamental concept is scope, the challenge is then choosing it appropriately so that strategic benefits can be achieved. The business side can easily have their focus on a particular problem of particular LOB that can lead to a bottom-up approach. Likewise, IT has a cross-cutting position that can allow it to take a broader view, seeing things from a top-down perspective, or vice versa. So how does an organization go about choosing the right scope? The key is understanding the business. What makes this difficult? In my opinion, it is projects. Why? Projects, as soon as they are defined, automatically limit the scope of the analysis. Trying to perform analysis that goes beyond the scope of the project is the worst nightmare for a project manager. From a bottom-up perspective, analysts need to try to get away with as much as they can to understand the business beyond the scope of the project at hand. A business process analysis approach can help do this, and can be justified in the name of creating a more usable, appropriate solution. This will help move us up a bit from the bottom. At the same time, organizations need to be performing analysis without any particular solution in mind. Akin to a top down approach, an organization can simply attempt to document business processes at a department or division level and wait for the results to determine what projects should be put in motion. This allows them to focus on opportunities for strategic improvement, rather than continue to put band aids on current pain points.

The gist of this long post is that BPM and SOA are inseparable. Likewise, any enterprise adoption will involve both bottom-up opportunities and top-down opportunities. By managing those efforts and ensuring that everything keeps going in the same direction, your chances for success will go up.

I recently posted on being an internal service provider. I’m happy to say that courtesy of Jason Bloomberg and Ron Schmelzer at ZapThink, you can now hear a lot more of my thoughts on the subject. Last week, we recorded a podcast entitled Taking a Product Management Approach to Services. You can hear quite a bit about the concept of product management that we practice at my day job, and how that experience is providing benefits as we embrace SOA.

I’d like to thank Jason and Ron for the opportunity, it was an enjoyable experience. James McGovern is always asking enterprise architects to blog, speak at conferences, etc., and I’d like to second that call. I don’t know if the ZapThink guys plan on making a series out of this, but personally, I’d love to see it- not just with me, but with a number of practicing enterprise architects.

The podcast-based interview provides an excellent forum for discussion and an opportunity to share knowledge with peers in a format that is far more convenient (and, as Jason & Ron point out, vendor free) than the typical industry conference. I have a long history with the ZapForums, beginning when they were done in a live format. Any of you who followed them know me as the regular listener who always had a question for the presenter. When they switched to a podcast format to reach a broader audience, Jason and Ron were kind enough to offer an opportunity to record one, and my employer was kind enough to give me permission to do so.

Please give the podcast a listen, and don’t hesitate to drop me a comment here or send me an email if you have further questions!

I was a bit annoyed when people suddenly stopped saying S-O-A and started saying so-ah, or as I like to point out, “So, uh…” That was only a pronunciation change. The worst that could come out of that would be kids in the National Spelling Bee asking, “Are there any alternate pronunciations?” (It was on in our corporate fitness center during my workout today…)

Well, if you haven’t heard, Gartner and Oracle have both starting using the term SOA 2.0. With some liberal pronunciation from a French descent, that now yields, “So, uh… duh!” That about sums it up. Duh! We don’t need “2.0”. Go sign this petition and stop the madness.

Dave Linthicum had a great quote on his eBizQ blog:

Integration should be a configuration exercise, not a programming exercise, especially when considered in the context of a SOA.

While it may be true that some integration still needs to be programmed, we should be striving for configurable integration. As standards are adopted and incorporated into more and more systems, we can get there.

There’s a lot more to Dave’s statement, however. If you read between the lines, the message is that to really achieve the agility touted by the SOA pundits, the things that need to be touched when a business change occurs should be configured, not coded. Coding is a longer exercise than configuring, in most cases. In order to reach this goal, tools will need to continue to improve. At the same time, we can’t lose sight of the importance of testing. The promotion cycles associated with the deployment of a coded solution typically require extensive testing in order to enter an environment. In the case of a configuration change, that may not happen, but in the future, the impact could be catastrophic, as ZapThink analyst Jason Bloomberg called out in his ZapFlash, SOA Governance and the Butterfly Effect.

The point of all of this is that if you are an Enterprise Architect leading an adoption of SOA, you need to make sure that you’re making the appropriate things configurable, geared toward your operations staff, and using your developers for the most important things, developing business services.

Congratulations to my friend, Brenda Michelson, on her new blog on eBizQ, Business Driven Architect. Brenda is the “architect-in-residence” for the Patricia Seybold Group, and was kind enough to appear at a workshop in St. Louis a few months back, presenting some of her research on SOA.

Her most recent posting lists some practitioners in business-driven architect positions. A few of these are on my blogroll, while I hadn’t heard of some of the others. Now I have, and have added them to my subscriptions. Personally, these practitioners are the ones I enjoy reading the most. Perhaps it is because I’m a practitioner myself, so the issues they discuss hit home. So thanks, Brenda, and I look forward to hearing what you have to say!

James McGovern asks why more enterprise architects aren’t speaking at industry conferences? As an enterprise architect who has presented at an industry conference, I offer the following opinions:

• As James suggests, the conference chairs don’t have a good network of practitioners. I established my contacts not through my employer, but through participation in the online community on my own time. Those enterprise architects that don’t do this go unknown. I, like James, would love to see more participation from our peers.
• On the other side of the equation are the enterprise architects themselves. Rightly so, their primary interests are in helping their company. Their thought leadership needs to be directed internally, first. This is the same reason why you’re far more likely to see a vendor, consultant, or analyst quoted in the press than an enterprise architect or other corporate practitioner. Unless you happen to be one of the lucky few who work at a company with a very liberal external communications policy, it can be very difficult to get approval for outside communications.

I consider myself very fortunate to have had the opportunity to speak at industry conferences in the past, and in the future. I am a strong proponent of giving in order to receive. The more willing I am to share my knowledge, the more likely people will open up to me and share their own knowledge. I’m of the opinion that most companies have people just as smart or even smarter than myself, and therefore, that means they’re probably thinking about many of the same things that I am, at least from a technology perspective. It’s not a competitive advantage situation if we’re simply discussing the merits of open source software versus commercial software; Java, C#, and Ruby; etc.

Thank you Phil Wainewright! Phil recently posted a blog on ZDNet titled: “It’s the user experience, stupid”. It’s nice to see the “long-neglected backwater of usability” getting some well deserved attention.

The Business Week article that Phil mentions was particularly interesting because it mentions “the natural advantage” that companies like SalesForce and NetSuite have because of their ability to track all activities going on in their solutions. So what can we learn from this? Is this about usability? In part, it is. While I’m sure these companies are performing usability tests, nothing beats actual field observation when it comes to usability. In the setting of a usability lab, if you’re fortunate enough to have one, you can glean a lot of excellent information. Unfortunately, it’s still a lab setting, and is largely dependent on your ability to get representative users. Unfortunately, we also can’t go out on observe each and every single user.

In the absence of field observation, the next best thing is to collect as much information we can from the solution itself. If you do this with a desktop solution, the spyware police will be all over you, regardless of how good your intentions were. If you host the solution, all bets are off.

So how does this tie into SOA? It ties into SOA for two reasons. First, SOA is about taking a more granular approach to the entry points into our solutions. By decomposing the problem down into independently maintained pieces, additional entry (integration) points have now been created. Every entry points represents a new location to collect metrics. Secondly, if you choose to use XML to represent the messages for your service interchanges, the opportunity exists to glean information from those messages (all the more reason to make sure you’ve encrypted all elements of the messages for which privacy must be maintained). Through analysis of this information, we can learn more about the service consumers and their usage patterns. Voila, there’s your field information. Need another example? Head over to IT Conversations and listen to Nathan Eagle’s presentation from Where 2.0 on what information he was able to glean simply by monitoring the location of cellphones, and the cellphones in proximity that were detected via Bluetooth.

Now there are obvious privacy concerns with the collection of any information. There are also clear benefits to the collection of metrics through management infrastructure. The key point of all of this, however, is that the better you are able to understand the needs of your customers, whether they are humans sitting in front of their web browser, or other systems issuing web services requests, the better solutions you’ll be able to provide. It’s not simply about collecting the information, but about collecting and using the information to make things better.

In between the recent (and completely unnecessary, in my opinion) debate over Web 2.0 and SOA, a more pessimistic tone toward SOA has begun to emerge. Joe McKendrick recently discussed this in his ZDNet blog, “Is SOA ready for the ‘slope of enlightenment?'”

The pace of things that the press, analysts, and vendors like to push is absolutely ridiculous, and increasingly frustrating. Unfortunately, it’s the way things work. Simply writing this blog has given me some exposure to this world. It is no easy task to continue to come up with interesting SOA related things to discuss on this blog. Kudos to bloggers like James McGovern who are posting every single day, and on the whole, keeping it interesting. I’m doing it on my own time, with only a goal of sharing information with experts in the community. Imagine if I was actually trying to make a living based on this information. In order to keep it interesting, you either have to provide a new spin on the same old information, or take the emphasis off of the information and focus it on the presentation. The most successful path is to do both: continue to look for a new angles (and new information), while doing so in a manner that keeps people interested.

On the flip side, there are those of us that are actually trying to adopt SOA within an enterprise. What’s a realistic timeframe to do so? It certainly isn’t 6 months, and I’d be hard pressed to believe that it can be done in a large organization in 18 months. Let’s face it, unless you’ve already taken some of the first steps toward SOA prior to the recent hype cycle, we’re talking about an effort that will take years, not months. It’s hard to market something successfully for that long of a timeframe. A case study that states that a company is 40% of the way there is less interesting than one that makes it appear that a company is 100% of the way there.

As companies are in the middle of their adoptions, comments about the “trough of disillusionment” actually make it more difficult for organizations to be successful due to management by magazine. The marketing hype continues the trend of unrealistic expectations, too much focus on short term gains, and a lack of strategic planning. For once, it would be nice if technology adoption had expectations like the drug discovery process. It takes years, often more than 10, to bring a successful drug to market. When a potential lead compound is found by a company researching cancer cures, there is no huge marketing hype. The lead compound merely goes to the next stage in a long, long process. Only when some form of clinical trial success begins to occur does the marketing machine come into play.

While I’m certainly not advocating a turtle’s pace in SOA adoption efforts, I am advocating a steady, managed approach. There are technology trends that will provide short term benefits, and there are technology trends that are broader and more strategic in nature. A company needs to leverage both to successfully use technology for competitive advantage. Just because it is technology-based, doesn’t mean that it can be implemented faster than any other strategic initiative.

A friend of mine, Fred Domke, blogged about using a B2B scenario as an SOA pilot. In a previous posting, I discussed my opinions on what constitutes a good pilot. The key item that I mentioned was choosing something that will expose the organization to the cultural changes associated with becoming a service provider.

In this case, I don’t agree that a B2B scenario makes for a good pilot. I agree completely that B2B scenarios are great applications for SOA, but I don’t think they’re going to introduce an organization to the cultural changes. The first problem is that a B2B integration may not involve the cross-organizational concerns that arise. Of the case studies/presentations I’ve seen, the B2B integration scenarios only involved one organization (at least on one end of the integration). That organization was already acting as a service provider, so there is no culture change needed. Yes, there may be some inefficiencies in the integration processes, but outside of some incremental improvements, will the use of standards-based integration technologies in this scenario provide the results for an adoption of SOA across the entire enterprise?

I think there are far more problems on the inside of the firewall that require the cultural changes associated with SOA for resolution. Companies that are doing business together already know that they must work together. Ironically, sometimes it’s easier to get two companies to work together than it is to get two departments within a company to work together. By all means, apply SOA to your B2B integration scenarios. You’d be nuts not to. But to really see the benefits, it needs to be applied inside the firewall.