Todd Biske: Outside the Box

Archive for the ‘SOA’ Category

I want to turn on Donald Trump next Sunday night and see him task the teams with the successful creation and marketing of SOA within an enterprise. Okay, so it can’t be done within the day or two that they normally have, and outside of some Dilbert-esque quotes, it probably wouldn’t make for good TV. What it would do, however, is allow IT to see what their culture needs to be like in the future.

There’s a discussion just getting started in the Yahoo SOA group that raises some questions about the importance of marketing in SOA. A frequent complaint in the boardroom on “The Apprentice” is that the marketing strategy didn’t cut it and as a result, the person responsible for marketing on that task is fired. IT isn’t made up of bunch of people with MBA’s from Harvard, Wharton, or even Trump University. I have two degrees from the College of Engineering at the University of Illinois. During my stay there, I was not required to take any marketing courses, although the Computer Science department did require students in their undergraduate engineering program to take 4 courses outside of the department (independent of other electives) to which CS could be applied. The most popular area was business, with my choice, psychology, being second. The typical techie does not have formal training in marketing or other aspects of running a business, so it’s no surprise that we have a hard time with it.

We need to bring some business savvy into the IT department. I’m not talking about an understanding of the business being supported by IT (although that’s important too), I’m talking an understanding of how to succeed in business. Marketing, sales, product development, research, etc. A service provider needs to think of themselves as a vendor. They need to have a customer centric focus, with an understanding of the market trends (i.e. the business goals), customer needs, product lifecycles, resource availability, etc. to be successful. IT cannot simply be order takers in the process, because technology usage within an enterprise is not a commodity. The business side can’t simply go to IT Depot at the nearest shopping zone and pick up what they need. There are elements of technology that can be, and this will continue to fuel SaaS and other managed services, but here and now, the need for the IT department still exists. It’s time to change the IT culture, and get the development teams thinking about Service Management and a more business-like approach to their efforts.

Update: While the whole idea of bringing MBAs in was somewhat in jest, this is exactly what IBM did. Joe McKendrick’s eBizQ SOA in Action blog brought it to my attention, here’s a link to the original eBizQ story.

Recently, there’s been a few posts that have discussed the role of the individual in an SOA effort. David Margulius of InfoWorld posted an article about a doctor that took it into his own hands to create an electronic records system for outpatient services. Joe McKendrick of ZDNet and eBizQ followed on with some commentary. Prior to this, there was an article by a developer that was particularly critical of Enterprise Architecture.

Both of these articles raise some interesting points, and for the discussion, we need to go back to the city planning analogy that I recently discussed. If we were to equate these two technology related activities back to the city planning analogy, what would we have? In the case of the doctor that built his own electronic outpatient records system, let’s look at a recycling program. It’s entirely possibly that a homeowner’s organization could use some of the dues collected and implement a recycling program for an individual subdivision. It’s even possible for an individual homeowner to take their recycling to some provider elsewhere. Does it work? Absolutely. Is it cost-effective at the city level? Well, maybe not. Every individual getting in their car and driving to the nearest recycling center will be more expensive than a few trucks coming from the center for curbside pickup. The homeowners organization may be slightly more cost effective, but if each subdivision picks their own provider, you could have problems with far more recycling trucks on the street than are really necessary. The real concern, however, comes back to the problems in the first place. Why didn’t the city council put a recycling program in place? The fact that the city failed to act on this in a timely manner for its citizens is what causes a homeowner or homeowner’s organization to take matters into their own hands.

Likewise, let’s take the case of the frustrated developer. Here, the author called out:

Projects are driven by business deadlines, and not by the time it will take to get things right. So the foundation starts out weak…

This sounds like a case of being doomed from the start. In the city planning analogy, there are many scenarios, ranging from fixing a pothole in a street, to building highway interchanges. How many stories have been there about some worker that follows the letter to the T? They’re told to fix a pothole, so they fix that pothole, and only that pothole, regardless of whether or not the street looks like the surface of a golf ball. Some major development project is put under way, and winds up having huge cost overruns because an over-aggressive schedule.

While some may think that the core of the problem is too much governance, it’s not. The problem is ineffective governance. There needs to be an understanding from top to bottom of the roles and responsibilities associated with decision making and all need to be held accountable. A developer that isn’t following the guidelines isn’t a good thing, nor is an architect that establishes the wrong guidelines. An enterprise architect that obsesses about particular lines of code in a project is focused on the wrong thing. The city council shouldn’t be mandating what color a homeowner paints their bathroom. At the same time, there will always be gaps, and there will always be people with time on their hands to address them, like the doctor in the InfoWorld article. A key to being successful is the creation of the appropriate framework so those people can fill those gaps, but in a way that leads not only to short term success, but long term success, as well. I’m just as guilty as anyone else of jury-rigging a solution for a problem that I had, and then telling a few people about it, who in turn told their friends about it, etc. and the time I spent on it starting going up and up. The problem was a lack of understanding of either the need or the value that would achieved from the solution.

SOA is just another example of this. I recently had a discussion with Phil Windley about an upcoming article. In our discussion, one of the things I mentioned was that I felt that the technology governance (i.e. EA) needs to get in sync with the traditional IT governance (i.e., project scoping and funding). Project establish boundaries, and if those boundaries make it difficult to build services that have broader benefit, the odds are already stacked against you. We need to find a way to have those services be built the right way, and that may be more about changing the way that IT operates than it is about using a particular technology.

In short, the groups that comprise IT Governance in my opinion, EA and Management, need to establish the right guidelines and the right funding models for SOA, and really IT as a whole, to be successful. For some larger efforts, it will involve significant planning and effort from those areas. The smaller efforts will still occur, however, and they can’t be ignored either. An environment that encourages the individual to contribute to the overall success, and helps them to contribute in a constructive, rather than destructive way, will be the most successful. Sometimes it is necessary to deny a building permit for the greater good. The best situations, however, are where it can be turned into a win-win situation for both.

Dan Foody of Progress Software has a great post entitled “The SOA Glass Ceiling” that brings up a familiar topic- the service lifecycle. He states:

If you only take away one thing from this blog its that in SOA you don’t build services, you deliver services.

What was particularly interesting to me was the discussion around product delivery. I’ve always used the analogy of a product manager when discussing the service lifecycle, and it would seem that Dan’s article shoots a big hole in this analogy. In reading into it, however, it doesn’t. Dan describes product delivery as this:

…you roll out a service, and at a later date (6, 9, 12 months later) you roll out the next version, and the software development lifecycle continues.

Inside of most enterprises, I’d actually argue that this is an improvement. Because what’s really happening is that all of the efforts are disjoint. Most likely, there may not be a next version, and when there is, it’s a completely new set of people that lack the context of the original team. The management was project management, not product management. Dan is in agreement, as he states:

IT organizations are used to building applications for the business, shipping them, and going on to the next (typically) unrelated thing to build.

I like the notion of Service Delivery. Obviously, building and deploying the service is a part of it. But to do it properly, the entire Service Lifecycle must be managed, and it’s not strictly about project management of the development efforts for each version. It’s also about observing how customers are using the service, marketing the service out to the enterprise, providing excellent support when expectations are not met, establishing contracts to ensure expectations are clear from the beginning, and much more.

Brenda Michelson of Elemental Links just posted a great excerpt of an unpublished paper from Annie Shum of BEA that discusses the city planner analogy to SOA.

Interestingly, my recent post on facilitating the service lifecycle spawned an email conversation on this topic. The gist of the conversation was that many of the tools out there in the SOA space are targeting the Building Inspector, rather than the City Planner or the Developer/Landlord (developer in the city planning sense, not in the software development sense).

Annie’s paper points out that:

City planning is not concerned with the design and construction of individual buildings. Rather, it is concerned with the multi-aspect relationship of individual buildings to one another… city planning is not about building architecture; it’s about meta-architecture for designing communities of buildings by focusing on common infrastructure, governance and cohesion.

This is a great statement, and very useful when trying to explain the role of an enterprise architect. James McGovern recently brought up the need for architects to work downward to the developers. I think the city planning analogy can be a useful tool in helping to create an understanding of the roles involved. Interestingly, recent events in my town demonstrate how well this analogy works, even in providing some guidance to how collaboration should occur. The town I live in is growing. Years ago, it would have been considered a rural farming community. Now, with the normal suburban sprawl, its close proximity to downtown St. Louis is making it more suburban and less rural. The biggest event associated with this growth is about to happen: a major tract of farmland is going to be turned into a major shopping and industrial complex. The opinions page in our local paper has become very active on the subject. The individuals expressing their opinions are akin to the individual developer in the enterprise. The city council is akin to the enterprise architecture team. I would not want to leave this effort in the hands of the individuals who are expressing their opinions. They simply don’t have the right perspective to make these types of decisions. The city council has been working on this for at least 5 years, studying the implications of tax incentives, infrastructure development, impact on local businesses, etc. The roles are clear. What I do believe in, however, is the ability of the individual to participate and contribute to the process, and the responsibility of the city council to solicit input and communicate their direction. After all, this entire process may have begun with an individual land owner selling their acreage to the developer to begin with.

The same holds true for the corporate enterprise architect. While enterprises are not democracies, so your EA can’t be recalled or replaced at the next review cycle by a vote of the developers, the best practices of a democratic style of government should be followed. The Enterprise Architect must be producing documents for upward communication (business and IT executives), lateral communication (other architects), and downward communication (developers, operators, engineers). Likewise, they should be soliciting input from all three directions, as well. Communication should never be one-way, nor should it be one size fits all. Some of the best ideas may come from people in the trenches, and some of those people may be a future architect. I was one of those people in an organization some 7 years ago. Annie’s paper points the importance of communication and collaboration with this statement:

…the entrprise architects’ role extends well beyond software coding specifications. Instead they must also act as the chief enablers of borderless collaboration by coordinating and prioritizing the input from disparate groups with different needs, interests, and views, including business stakeholders, software architects, developers, and DBAs, as well as external partners, suppliers, providers, customers, auditors, and so on.

Brenda pointed out that this document hasn’t been published yet. Based upon this excerpt, I hope it will be published. Add me to the list of interested parties.

I’m listening to the latest Technometria podcast from IT Conversations where Phil Windley and his gang have a discussion with Ben Galbraith. At 53:45 into the podcast, Phil asks Ben about his home automation efforts that he’s incorporating into his house under construction. While there’s no mention at all of SOA, it was an interesting discussion nonetheless. He’s planning on leveraging RFID, webcams, etc. It sounds really cool, and I’d be interested in hearing an after-the-fact case study on the integration problems he ran into. Ben’s a great guy, I had the opportunity to meet him at a Microsoft Technology Summit in 2005 (he probably doesn’t remember me, but who cares), and knows a tremendous amount about the presentation side of development technologies.

Do you have ERS? Jeff Schneider had a great post outlining the symptoms of Empty Registry Syndrome. One of the things that struck home with me was his last suggestion on how to cure it.

Verify that your registry doesn’t stink. Here’s the test: If you search for a service that doesn’t exist does it return with:
A) No results found
or
B) No results found, would you like to request a new service?
If the answer is “A” please call your vendor and let them know that their software is spreading a disease known as ERS.

Prior to becoming a consultant, I was at an enterprise where we were just beginning to explore the registry/repository space. We had one vendor come by and give a demo, and I asked them, “Is there any way to put services into the registry/repository that are in a planned state?” “Umm…. that’s a great idea!” A couple months prior to this, a project team contacted me about a service they would need. No one had implemented it yet, but we sent out a notification to the development teams about this request, and in came flooding all sorts of requirements, with no more effort than an email. It’s amazing what a little bit of communication can do, and think of how that effort could have been facilitated by some tools. The enterprise was embracing workflow technologies. I’m starting to see some potential here.

1. The project architect is building out the candidate architecture, and as that happens, the service library is searched.
2. If there’s a match, the architect can immediately register interest in the service. There may be tasks spun out of this to either the project architect or the responsible service manager.
3. If there isn’t a match, the architect has two option. Request that a new service be created, or create a planned new service.
4. If they request a new service, this results in tasks for a governance group to determine who the right owner should be, and schedule out a project. The project architect of that new task would update the service to a “planned” state from a “requested” state.
5. For services that enter the “planned” state, notifications would go out to the appropriate development teams with pertinent information, allowing them to contribute to the service requirements.

This scenario that I’m painting is an area that I’m going to call “Service Lifecycle Management.” Governance is a part of it, as shown, but there’s much more to it than that. It’s also not SDLC management, either, although that too is part of it. The service lifecycle doesn’t end when the service has been developed. The service lifecycle only ends when the service is decommissioned.

I think this area is one that is ripe for opportunity. I commented about this last year, but this topic still hasn’t received much attention from the vendor community. Mindreef was heading down this path back in Dec. 2005 with their Coral product, but there’s no mention of it on their website anymore (perhaps it was incorporated into SOAPscope Server). The challenge with lifecycle support for vendors is that it has to be about integration. Any product that tries to own the entire lifecycle will probably struggle. I see this space as being a combination of workflow/bpm, registry/repository, service management, and a service lifecycle management tool, at a minimum.

Look for a more in-depth blog on the Service Lifecycle soon. In the meantime, if you’re interested in an introduction to my thoughts on Service Lifecycle and being a Service Manager, a good starting place is this podcast I did with ZapThink last year.

James McGovern posted a question for me to answer:

“…crisply define the distinction between consulting firms who offer services that are consumed by enterprise architecture teams vs. the actual act of practicing enterprise architecture?”

In thinking about this, my first thoughts were pretty simplistic. Unless the consultant is in a long-term staff augmentation role, the consultant assists, the corporate EA does. That’s painfully obvious, however, and I don’t think it’s what James was really asking about. As I thought about this, it really came back to collaboration. In order for a consulting firm to be successful in the long term, it has to leverage the breadth of exposure it has. Within an enterprise, the opportunities for cultural breadth (corporate culture) are minimal. It may happen when a change in management is made, or when a merger or acquisition occurs, but in general, you settle into one way of doing things. A consulting firm, on the other hand, gets exposure to many corporate cultures, and has to find a way to make an organization successful within that culture, potentially being an agent for change. An 8 week or 12 week assignment is typically not long enough to introduce significant cultural change, however.

Given that the consulting firm has the breadth of exposure, it is critical that the consulting firm collaborate on engagements so that the best solutions can be offered. Even if I’m the only person assigned to a particular engagement, I will bounce ideas off of my peers to get their thoughts. By seeing what different organizations do, and understanding which approaches work best with a particular corporate culture, the engagements are much more likely to be successful.

Within an enterprise, I don’t think the notion of collaboration is as important as it should be. All too often, it’s about turf battles and clear lines of responsibility. When I ask my teammates a question, I don’t have any fear that they’re going to get recognition by my boss before I do. I ask them because I want the best solution for my client. In the enterprise world, internal advancement and recognition is on the mind of many individuals, and may come at the expense of creating a collaborative, team approach.

All of this is not specific to enterprise architecture, but I think it’s even more important at this level. In my experience as part of an EA team prior to becoming an enterprise, it became very clear that architecture couldn’t be done in a vacuum. Yet, the corporate culture makes it very difficult to foster the type of collaboration that was needed to make it successful. Too many meetings, too many fire drills, and too many time-dependent activities (i.e. top priority is some date in a project plan, not necessarily getting the best answer). This isn’t a knock on my former employer, it’s probably true of most enterprises. Enterprise architecture, by its very nature, requires collaboration. The EA focused on Security has to be working closely with the EA focused on Information and the EA focused on networking technologies, etc. Ironically, what often causes that collaboration to occur quickly? Bringing in a consultant. The consultant needs to quickly understand where an organization is, and it meets bringing all of the right people together and having the right discussions.

James McGovern recently posted on the subject of self-evaluation. I’m guessing that it must be annual review time at his place of employment. He makes some generalizations that I agree with, stating that it is “human nature to either trivialize and/or underestimate capabilities in other folks that they do not possess.” I believe the opposite is true as well: people tend to over-estimate and over-emphasize capabilities that are similar to their own.

While James’ chose to focus on how ineffective the typical evaluation and assessment process is for employees, I’d like to take it a different direction. While I agree that most people aren’t very good at assessing others, and probably even worse at assessing themselves (it can go either way, overly negative or overly positive), that doesn’t mean the practice should be abandoned, particularly that of the self-assessment.

One of the things that I’ve seen stated in articles and books, as well as experienced first hand in discussions, is the relationship of a solid self-understanding to being successful. You need to know your strengths and you need know your weaknesses, and you need to be objective about them. The successful leaders don’t go out and find people with the exact same strengths as their own. They go out and find people that complement their strengths. For example, I consider myself a big picture thinker, probably a common trait among architects. At the same time, I recognize that I’m not a very detail-oriented person. While I can certainly be an active participant in a project-planning workshop putting my fair share of post-it notes, seeing the details does not come natural to me. For others, the exact opposite is true. They are able to see all the i’s that need to be dotted and all the t’s that need to be crossed, but they may not be able to tell you the paragraph that’s being written. So, when I’ve been assigned to a project in a leadership position, the first thing I always request is that the stakeholders find someone who is very detail oriented to work with me. While I’m sure that person will drive me nuts at times, the project won’t be successful without it. Do I continue to learn and improve? Sure. A key to success is to understand your strengths AND your weaknesses, and surround yourself with people who can complement you.

The other important thing that needs to be called out, which James did in his blog, is the importance of objectivity when performing an assessment. Well, what is objectivity? The truth is, there’s almost always going to be some amount of subjectivity associated with any assessment. Where does this subjectivity begin from? It will usually stem for the reference model that is used for comparison. If this reference model is documented, and both parties agree to it beforehand, you’ve at least eliminated one variable from the equation. The less that is formally documented, the more difficult the assessment becomes.

For example, if I were to come in and perform an SOA assessment on an organization, I need to have a reference model that I compare my findings against. There’s a problem, however, if the things that I deem important aren’t the things that you deem important. The process must begin with a mutual understanding of the things that will be assessed, and the criteria that they will be evaluated against. In my previous job, I was discussing a potential assessment with some vendors (consultants and product vendors), and some of them wanted to cram their model down my throat, even though it would have focused on a number of things that I knew we hadn’t done anything with yet. That would have been a waste of my time and theirs.

So how does this work for self-assessments? Again, you need to have a point of comparison when you evaluate yourself. If your reference model is out of whack, your assessment will be too. I don’t want my daughters or my son growing up and comparing themselves to what the media may portray as “important” for young women or young men. That’s a problem with the reference model. This reference model has to be independent of what your boss wants or what your company wants. Why? Without it, how will you know when it’s time to make a change? Your company has its own needs and desires. You have your own needs and desires. The best possible situation is where both needs are met. That won’t always be the case, however, that’s why people (should) change jobs or even companies. If the things you find important don’t match what the company finds important, that doesn’t mean you are a lousy employee or that your company is lousy. It just means that the needs don’t match and it’s time to make a change. That could mean a move within the company, or it may mean leaving. You can only know this if you’ve got a solid understanding of what you want to do, when you want to do it, and whether you’re capable of doing it. To do this, you have to begin with your own assessment.

My final piece of advice on this is pretty simple. Anytime you do an assessment, you should strive for some amount of balance. There should always be things that you want to call out as well done, and there should always be things that you want to improve. If everything looks rosy, or everything needs improvement, then you’ve got an unrealistic view of yourself.

Joe McKendrick had a post on January 17th entitled, “SOA reaches out to the cloud – will business follow?” He discusses Dion Hinchcliffe’s prediction that in 2007, SOA will open up more to the Internet cloud. Dion had cited a McKinsey and Company survey that stated that 48% of CIOs surveyed are planning to open their SOAs “to the cloud” in 2007. Joe inquires as to what this means to the SOA business case.

My own thought is that it’s not about the SOA business case. SOA doesn’t justify opening services up to the cloud. It’s the other way around. If the business needs to communicate with external parties in a system-to-system interaction, guess what, you need SOA. I don’t think there’s anything new here, and clearly, many of the early SOA efforts that have been presented at conferences reflect this.

A friend of mine used to work for Rockwell Collins and had presented on their SOA efforts, which largely was around providing Web Services to their corporate partners who previously required a person to log into an extranet portal to interact. The business hadn’t changed. Their business partners wanted to take some of the human element out of the business processes in interacting with them, and to do so, needed services. One could even argue whether this was really SOA or not. The system that they have today, clearly is a component of an enterprise SOA, but obviously, there were not services behind their existing portal that were simply opened up. They needed to rip apart the existing system and expose services.

So, my own opinion is that I’m not all that excited about this prediction. Companies need to do business. Often times, that business requires interaction between two or more companies. Can SOA help in that interaction? Absolutely. On the flipside, however, is SOA really opening up new interactions between businesses that previously didn’t exist, or is it simply allowing those interactions to be more efficient? There may be some smaller opportunities that have gone unnoticed. A friend of mine, Fred Domke of Business Integration Technology, brought up the subject of office supplies over a lunch. Every large enterprise needs office supplies, but how many of them have optimized the supply chain with it, leveraging SOA and BPM technologies? It’s probably not something that’s high up on any CIO’s list, but I’d bet that there is some potential savings there.

I have previously posted on the topic of outsourcing and said that SOA doesn’t necessarily mean any more or any less outsourcing, but it should mean a higher rate of success. Organizations should have a better handle on the boundaries that make up their systems, and as a result, have a better handle on where the key integration points are. At the same time, this effort could open up new opportunities, but I think those will be driven by the business strategy, not by the technology.

I had a meeting with a vendor earlier today and at one point, he said in jest, “I’m sure you’ve got some web services around your house.” I replied, that I didn’t, which I don’t. But I certainly thought that there’s plenty of opportunities for it. I was just going to leave it at that, but then I got home and the first entry in my news reader waiting to be read was a story from The Unofficial Apple Weblog on the Indigo Home Automation and Control Server. I decided then it must be a sign.

First off, there’s absolutely no reason SOA can’t be applied in the consumer world. I’ve previously blogged about SOA for schools, but this is the first time I’ve thought about SOA for the home. I don’t have any of the home automation technology in my house (i.e. X10), but I’m at least aware of it and what it can do. While X10 is definitely a de facto standard, why shouldn’t all the device in our home be web services based? Now, it probably isn’t cost effective to embed a SOAP stack in every lightbulb. Besides, X10 works just fine for that. There are plenty of integration and process problems in the house, however. Why do you think that convergence (i.e. integration) has been a theme of CES for the last 5 years? If all of the devices in our house had a standardized interface and spoke a common language, there could be great potential. I’d love to have a programmable oven that I could push cooking instructions to from the recipe I’ve got online. What about pausing the TV or at least turning down the volume when the phone rings? How about a programmable thermostat that actually taps into the various weather services available to really optimize power consumption? The same could hold true for an automated sprinkler system.

There’s a ton of research dollars that is being poured into the “digital home” right now, but how many of the vendors doing so are actually thinking about it as a service oriented home, with open standards and open technology? Unfortunately, the focus right now is on media, which means DRM, which means closed and proprietary. The AppleTV is a nice device, but the piece that missing, in my opinion, is the ability to stream to any monitor in my house. I’m not about to pay $299 for every single TV. Give me a $299 server, and some $50 access points to hook up to the various screens in the house, and now it’s beginning to look promising. We have two DVRs in our house, and it just sucks when I want to watch a show in our living room, only to remember that we had recorded it downstairs because something else was on upstairs. So, my advice to all of the vendors out there dealing in the digital home is to start thinking about the service oriented home, and giving the consumer open services to be able to do things the way we’d like.

An expression that you may have heard bantered about in SOA discussions is “design for change.” A recent exchange on a Yahoo group made me decide that I really don’t like it.

If you think about it, how can you possibly design for a change, unless you know what that change is going to be? In that case, you’re not really supporting change, you’re supporting a known quantity. The only example that I can come up with that actually makes sense is systems that deal with regulatory enforcement. For example, Intuit knows that the Tax Code changes every year. If they had some web services associated with this in their TurboTax products, it is possibly that the interfaces of these could be designed to support change. The reason they’re able to do this is because they have a good idea on how things change. They may not know what aspect of the tax code will change, however, so there’s still plenty of work to be done.

What we really should be espousing is a three-pronged approach. First, architect for change. Okay, some of you are immediately going to cry foul and say, “What’s the difference?” To me, architecture is about establishing boundaries. It is the process of splitting a problem up into independently maintainable components. To architect for change, you don’t need to know what changes will occur, you need to know where the changes may occur. If I haven’t broken out my tax calculation service from the user facing system that requires it, adding new tax laws into the product is going to be more costly because I did not establish appropriate boundaries. SOA is about architecting for change, not designing for change.

Second, design to the best of your current knowledge. In other words, don’t try to predict the future. Design is about what goes within those boundaries, but certainly does involve the boundary itself (i.e. the interface). If you try to come up with an interface that supports all currently known needs as well as trying to predict the future, you can run into all sorts of problems ranging from analysis paralysis to an interface that works for all but is equally hated by all. One point of note on this. Designing to the best of your current knowledge doesn’t mean you only accept input from service consumers currently under developed. If you know that another system is going to use your service in the future, then you should incorporate their needs into your design. If you hope that another system is going to use your service, now you’re starting to walk on thin ice.

Third, and most importantly, plan for change. It has been said that the only constant is change. Your systems, your services, your business- all of them will change. What gets us into trouble is not the fact that things have changed, it’s that it is a mad scramble when things do. Schedules have to be synchronized, resources assigned, etc. Think about how you deal with your vendors. Which would you rather deal with? A vendor that releases a version every 6 months, on schedule, without fail, or a vendor that sometimes releases versions within a month of each other, but sometimes as long as 18 months? As a consumer of that vendor, how can you possibly hope to plan your upgrades? To become a service provider, you must figure out how to effectively manage change. A standard release schedule for every service would be a great start. For an enterprise, those changes may not occur as frequently as needed for a commercial product. Perhaps there are standard dates that must be used.

The problem is not a technical one. This isn’t a debate over SOAP/WS-* versus REST. If the underlying XML message changes, the consumer and provider need to be modified, presuming the change isn’t backward compatible with the existing schema. If the organization has to scramble every time this situation occurs, that causes problems.

Continuing on my discussion about this Technometria podcast on IT Conversations, where Phil Windley and others chat with David Platt, author of a new book entitled “Why Software Sucks… and What You Can Do About It.”, there was a great item on the importance of instrumentation.

David gave the example of a web site for a library. He said how will read a review of a book online and immediately go to the website of the library to request it. He said that the UI is hard to use because there are two use cases: one where the user is in the physical library and one where the user is accessing the web site from home. When you’re accessing it from home, a user really doesn’t care if the book is in the library, where a user in the library certainly does. As a result, a design suited for use in the library doesn’t work well for people accessing it from home. David asked the team what percentage of requests for the site came from within the library, versus what percentage of requests came from outside the library. They didn’t know. Not knowing this makes it very difficult to design the site properly. David points out that you need to design for the masses and not the edge cases, although that’s so often what we do.

The importance of instrumentation has always been a soap box of mine. Back in May of 2006 there was a Business Week article that discussed the natural advantage of web-based companies like Amazon and Salesforce.com which all revolved around instrumentation (my comments here). I just recently had a posting on the power of the feedback loop. None of this can happen without instrumentation, and this doesn’t apply solely to user interfaces. Do you know which operations of your services receive the most use? Do you know when they receive the most use? Do you know where that usage is coming from? This type of information needs to be captured and fed back into the development process to create better services, and make better use of resources. If 99% of a service’s operations aren’t used, why were they built in the first place? Without instrumentation, how will you know this?

Here are two examples that I’ve seen first hand:

• A service that normally had about 10,000 requests a day, but every now and then, it would balloon up to 100,000 requests or more for two days. There was a consuming application associated with end-of-quarter reporting that hammered the service. While no problems were experienced, this could have been a disaster. This directly led to work to capture accurate usage profiles of new consumers before they went into production.
• Another service was seen to have spikes of usage first thing in the morning, over lunch, and at the end of the day. These were times when the users of these applications had time to sit down and use the application versus the other activities that went on over the course of the day. Adjustments had to be made to the infrastructure to support the spiked access pattern instead of a steady rate over the whole day.

These examples served to open the door to better instrumentation. Things that should be looked for as part of a continual improvement process are sequences of service interactions. I may see that two or more services are always called in sequence, by multiple applications. This may be an opportunity to create a composite service (or simply rewrite the first service) so it handles the entire sequence of interactions, improving performance, and making life easier for the consumer. You can only get this information through instrumentation, however.

Phil Windley, in his latest Technometria podcast on IT Conversations, had a discussion (well, he listened a lot) with David Platt, author of a new book entitled “Why Software Sucks… and What You Can Do About It.”

While the podcast is quite lengthy, it’s very good and quite entertaining. As someone with a background in human computer interaction and usability, David’s comments certainly hit home for me. While it wasn’t anything that I haven’t heard before, it’s still something that every developer should hear. Near the end of the podcast, he gave some guidelines for both users of systems and developers of systems. One thing he said that should be done is to have someone involved in the design of the user interface that has absolutely no clue about the implementation of the system. He gave an example where this person would ask the question, “Why are those two fields next to each other?” Often times, the response would be, “Well, those two columns are next to each other in the database.” Guess what? You’re letting the implementation drive the interface.

This is by no means an easy task. I had a conversation with a good friend of mine who is a consultant on user centered design and usability, where he was asked about some of the potential conflicts between a user centered design approach and an SOA approach. My comments were that there shouldn’t be. UCD should be concerned with the user interface, period. There will be backing services that support that user interface, but those interfaces are system interfaces, not something that gets exposed to the user. The conflict only arises when that separation between interface and implementation begins to blur. Everyone knows that people tend to take the path of least resistance, so it’s very easy to design a system where the user interface unduly influences backend implementation (and hence service design), and vice versa. The challenge is to realize that there is a separation of concerns. The UI team shouldn’t be telling the service developers how to do their job, and the service developers shouldn’t be telling the UI team how to do theirs. At the end of all of this, we somehow have to come up with a system that meets both concerns. It’s not an easy task, but it’s what makes our jobs fun!

Now, keep reading to my next post with another great item from this podacst. I didn’t want to include it here, in case people didn’t have interest in this part of it.

While there will be no shortages of blogs on this product, you can add my name to the list of people who can’t wait to get my hands on one. What a coup for Cingular, as well. Dana Gardner has already stated that he’s prepared to pay the huge contract termination fees with Sprint and Verizon to get one. I’ll have to change providers as well, although I’ll only be 6 months shy of my contract length, so I’ll have to do some math to see what the best option is. Hopefully, Cingular won’t be charging a premium for services on top of what the other smart phones may cost.

I’m sure that this device will have its fair share of glitches in the initial release, but it’s really refreshing to see this level of leapfrogging occur in the technology space. It’s even funnier after Bill Gates’ interview with C-Net that talks about how Apple is at a disadvantage due to their closed platform. While there are elements of truth in what Bill had to say, Apple is doing exactly what they need to do to do a successful, innovative company. While Microsoft may have a more stable revenue stream as the backbone of the Windows PC, they will be stuck in the world of commodity technology. While Robert Scoble calls out that Microsoft has competing technologies, it seems to be that it’s either in the wrong form factor or poorly marketed. I don’t want a gaming system to be able to stream video to my television. I’d rather have a device that excels at that function. I wouldn’t classify the AppleTV as innovative, but I would say that it puts user experience above all else, which is what will differentiate it, as with the iPhone.

I’ve been participating in a thread on a Yahoo group about Service Analysis and Design, and broached the subject of project management. I recently authored a paper on this for a client, and thought it was time to share some thoughts on this. Mark Griffin recently had a post titled SOA Mistakes where he discusses avoiding JBOWS: Just a Bunch Of Web Services. Why do companies wind up with JBOWS? I think a lot of it has to do with the project management culture of organizations.

What is a project manager’s primary concern? Well, it could be one of three things: delivering the solution as defined (scope), delivering the solution on time (schedule), or delivering the solution on budget (resources). Based on my own experience, the one thing that will drive a project manager nuts is when you add scope. Adding scope either lengthens the schedule or adds cost, both of which are probably the thing that the PM’s performance is being judged against. Most PMs I know aren’t evaluated on their ability to deliver what was promised, they are evaluated on whether dates were met and costs were controlled.

Now, the next assumption is tied to the project inception process. Odds are that the project was proposed because of some pain point- a symptom. It’s like going to the doctor and saying “I have a sore throat.” IT goes off and builds some user facing application to address that symptom, because that’s the information they were given, establishing the maximum scope of the project. Using my doctor analogy, it would be equivalent to the doctor simply looking at your throat and giving you a box of lozenges. Now let’s suppose that user facing application that we built required a new service. That service would be built solely to the specs of the user facing application. If you asked the project manager for resources to seek out other requirements, they’re going to tell you that’s out of scope. So, the service then becomes a part of the throat lozenge. In reality, the reason behind the pain point may have been a broader business process problem. It could also be the case that the pain point had become an epidemic and was also occurring in other parts of the enterprise. Using the doctor analogy, I could also have had a runny nose, with itchy eyes, and hives breaking out- symptoms of an allergic reaction. In this case the lozenge is going to do very little to alleviate the problem, just as the IT solution presented probably won’t do very much in the long run either.

There are two things I’d like to throw out there as suggestions that may help reduce your risk of JBOWS:

1. We need to be performing analysis outside of the context of project, solely for the sake of getting a better understanding of the business and the IT solutions that support it. If we don’t have models that give a broader picture, IT (and the business) will always be in the position of treating symptoms rather than the disease. If analysis is only performed within projects, the scope of that analysis is already constrained. Projects create constraints, just as architecture creates constraints. The architectural constraints should help drive the project definition, however, not vice versa.
2. Services should be broken out as independently managed efforts. I think the temptation to cut scope is simply too great when a consumer of a service and the service itself are developed within the same project. For sure, if a service is known up front to have more than one consumer, it must be managed independently of the development efforts of either consumer. Unless your organization is able to be flexible when adding scope, the risks are too great. If you want loosely coupled services, then the service development should be decoupled from the service development.

Avoiding JBOWS requires challenging the way we’ve done things in the past. If you’re still building projects the same old way that you have, odds are nothing will change. Services will be built within projects, and people will have to justify breaking a service out as a separate effort. We build to our constraints and assumptions, and if people assume it won’t be reused, they’ll probably build something that won’t be reused. Rather, you should take the opposite approach. Assume from the get go that all services will be enterprise services, and will be reused. If the organization is embracing SOA, people should have to justify why a service shouldn’t be independent, not vice versa.

I’d love to hear other people’s thoughts on this. I’ll be posting some more on the subject, specifically on managing the handoffs when consumers and services are being developed in parallel.