Todd Biske: Outside the Box

Governance is one of my favorite topics.  If someone asked me the thing that will influence the success of an SOA initiative the most, it would be governance. A very easy way to look at governance it is to compare it to a traditional government.  A government has to legislate, provide infrastructure, maintain strategic plans, enforce laws (police force), etc.  These are all activities that an IT organization must do to govern an SOA.  In reality, these are all things that an IT organization should have been doing, regardless of whether SOA is being done or not. 
 
The same challenges that municipalities face in their strategic growth are faced by IT organizations.  Urban centers grew through a very centralized approach, but have had to become more and more decentralized due to suburban sprawl.  As rural communities have grown, they have had to work more and more with their neighboring communities, possibly sharing common infrastructure and services.  The same is true of IT organizations. The urban center can be thought of as the mainframe or legacy systems.  Due to the web, web services, etc., portions of the legacy logic needs to be decentralized to meet the demands of the future.  At the same time, silo’d application development represents the rural communities.  These applications have grown, and the world of business processes is requiring them to work together seamlessly, rather than through inefficient handoffs and redundant processing. 
 
There are now tools out claiming to provide “SOA Governance.” The truth is that there is no tool or technology that will provide SOA Governance.  There are tools and technologies that can make governance easier, but ultimately, it will come down to process and communication.  If the process and communication isn’t there, the governance won’t be either.  At the same time, we can’t govern by process alone.  The things being enforced (i.e. the legislation) must be documented for all to see.  Herein lies the real challege with regards to SOA or, more broadly, applying governance to IT.  SOA is about looking horizontally while others are looking vertically.  How do you document the rules associated with making something an enterprise service versus an application-specific service?  Yes, we can have rules around WS-I compliance and naming conventions, but this often comes down to semantics and a strategic vision (i.e. business service blueprint).  This is akin to a business applying for a business license in a city.  There will be guidelines for the application that must be followed, but there is still a judgement that must be done by a city council as to whether they want the business in their city.  There may be general guidelines in the city master plan, and the opinions of the council members are exposed through the political process, but largely, things will be handled on a case by case basis by a set of people given the responsibility for making those decisions.  If you have the wrong people in place, you won’t be successful.
 

Most people are advocates of taking a WSDL-first/Schema-first/Contract-first approach to Web Services. I think most people would also agree that when implementing the service using Java or C#, the tooling support for this approach leaves a lot to be desired. That being said, we still need to do this.

The scenario gets more complicated when you take into account that many industry groups have produced data standards built using XML Schema. Many of these were established well before Web Services.

What I’ve been surprised by is the lack of documentation on how to bridge these two worlds. Maybe I haven’t looked hard enough, but I haven’t come across any best practices around XML Schema design when the intent is to use them within Web Services, especially when you know you’re going to be using Java or C# for the implementation. Anyone who’s gone the code first approach knows that class members map to child elements and not to attributes of an element. So, does this mean that when designing XML schemas for use within Web Services we should avoid using attributes of elements? If we don’t, does it mean that the developers writing the service are going to have to deal with low level XML APIs? Of course, a lot of this could be avoided by using a schema-oriented development environment typically associated with BPM tools, but that platform may not be appropriate for all scenarios.

Anyone with any pointers to some best practices or documentation in this space?

I was just reading this on MSDN. The author defines a contract as follows:
        •        A contract is everything about the behavior of the service that is engraved in code. They are drawn up at design time.
        •        Policies are everything that can be changed without coding. They are typically detailed later in the process and they are easier to change.

I agree with everything said here, except the use of the term contract. When I’ve heard discussions about service contracts, it goes beyond the functional interface as this article implies. Service contracts are by nature, defined not just by the functional aspects, but the non-functional aspects as well. Think about legal contracts in the real world. They not only say what is going to be done, but they say how it will be done, what the stipulations are if something is done incorrectly, etc. Applying that to system interaction, my tendency would be to rephrase the above definition as follows:
        •        The interface is everything about the behavior of the service that is engraved in code. They are drawn up at design time.
        •        Policies are everything that can be changed without coding. They are typically detailed later in the process and they are easier to change.
        •        The sum of the interface and the policies that dictate its usage constitute the service contract between a service consumer and a service provider.

All of this aside, this is a very good article detailing the importance of the interface, and the challenges associated with taking either a schema first approach or a code first approach to service creation. While I may not agree with his use of the term contract, I do agree with the fundamentals of the discussion.

Miko Matsumura writes in his blog:

I think as SOA “Crosses the chasm”, there’s a need for more people to think in terms of the architecture side. The architecture side tends to better keep an eye out for the bigger organizational goals. If SOA is truly to meet a larger vision including:
1) Aligning business and IT
2) Making IT strategic and a key part of competitive advantage
3) Enhancing business agility
4) Creating dramatic new forms of resource reuse
then folks need to be mindful of the architecture, the “A” side.

I agree with this completely. A favorite line of mine is that the “A” in SOA stands for architecture, not application. SOA is all about thinking outside of the box, that is, looking beyond the boundaries of the problem/project at hand. That’s something that doesn’t come easy.

At the same time, we can’t forget about the “S” either. The best solution will leverage the tactical “S” opportunities at hand to achieve the strategic “A” direction. There are risks in going too far in either direction. I consider myself more of “big picture” thinker and have always found that the most successful projects I’ve been involved with have had someone who was very detail oriented working with me. The “S” and the “A” must complement each other to achieve success.

While I won’t be there for this, I think Jon Udell has got some very interesting topics for the discussion, outlined in his blog here and here. For day one, a topic listed is “migration to policy-driven intermediation.” He mentions security and compliance policies, but I’d also add in operational (i.e. routing rules, caching, monitoring) and what I term integration (transformation, especially in support of versioning) policies. While externalizing these policies is problem one, what will quickly follow is a big management problem. Right now, policy management is typically provided as part of an enforcement solution. When those policies start to branch out beyond security, can the management solution still be sold as part of the enforcement solution? Is that what we want? Ultimately, we’d want the policy owners to be the ones setting these policies which could mean at least four departments: Information Security, Compliance, Operations, and the development team. How many tools currently support this scenario well?

He also suggests some topics around the “evolving role of the repository” including the human factors. I think this will be an interesting discussion as we’re starting to see some convergence between the human-oriented tools from vendors like Flashline and LogicLibrary and the system-facing UDDI registries from vendors like Systinet and Infravio.

One topic that I’ve been interested to hear about (I keep meaning to call David Linthicum’s voicemail line for his <a href=”http://www.soaexpertpodcast.com/”>podcast</a> on this one) is how to make Web Services with a “pull” invocation model a first class citizen. A service is a function that is performed on behalf of some external party. There are really two styles for this: a push model, where the consumer “pushes” the request to the service provider directly (easily implemented as SOAP/HTTP(S) and may or may not involve a response message) and a pull model, where the service provider “pulls” the request from some form of queueing system, i.e. JMS, MQ Series, or MSMQ. Think of this as the difference between a session bean and a message-driven bean in the J2EE world. The problem with the “pull” model is that it’s not a first class citizen in the Web Services world. This isn’t a notification scenario, so WS-Notification isn’t appropriate. Notifications tells you something happened, but the source of the notification has no expectation that some action will be formed.

The current state-of-the-art for handling this would be to use some form of intermediary (ESB, WSM broker, etc.) to perform HTTP-JMS bridging. Until we have an open wire protocol like HTTP for talking to queueing systems, that’s about all we can hope for. The problem with that is we now move out of the Web Services and into the MOM world. Depending on the implementation platform (especially if it’s not from the same vendor as your messaging backbone), the fact that the payload is a SOAP envelope may be completely invisible to the platform framework, resulting in a lot more work for the service developer. This topic doesn’t really fit with John’s other topics, but it’s of particular interest to me, so if you’ve got thoughts on this, blog away.

Overall, I think this forum should be very interesting. I look forward to following the blogs from it.

While I certainly haven’t been an active blog poster, I do have a lot to say around SOA and Web Services and now I’ve got my chance. I’ll be speaking at the ITPQ (IT Productivity & Quality) Conference on SOA and Web Services Best Practices in Chicago on October 18th and 19th. Here’s a link to a conference brochure. If anyone out there in the blogosphere will be there, let me know and I’d be happy to sit down and chat with you.

Aaron Skonnard, a proponent of contract first development, recently posted on the disconnect between what the vendors are doing and what developers want. He makes the claim that vendors don’t see the value of the contract-first approach. On this statement, I disagree. I don’t think the problem is that the vendors don’t see the virtues of it, I think it’s more the case that it is such a conceptual shift, that it’s a very difficult problem to solve when catering to an audience of C#, VB.NET, or Java coders. This is further justified by the fact that tools like BizTalk, as he mentions, make it so easy to do contract first development. The key is that BizTalk and most tools that fall into the Business Process Orchestration space are designed from the ground up for schema-oriented development. This makes working with XML messages and XSD much more natural. C# and Java are clearly for object-oriented development, so there’s a natural disconnect. There needs to be a bridge between the two worlds, and unfortunately, it began with an effort around traditional RPC mechanisms and as a result, wound up with hard, restrictive bindings between XSD and a C# or Java class. In my opinion, the toolsets need to bring some schema-oriented approaches to the tool sets for C# and Java. This usually involves code generation, and tools that do this have had a hard time gaining adoption. It also brings up memories of the whole object-relational mapping space, so the vendors are proceeding with caution. I expect that in 5 years we still won’t see advances in this area, but we’ll see the majority of services built using tools like BizTalk. For services that still need to be written in Java or C#, the developer will still have to deal with things using a DOM object, just as a database developer still often works with a RowSet. These will be relegated to more complex operations where low-level DOM access is a requirement.

Phil Windley recently asked in his blog, “Is WS-MetadataExchange really necessary?” I would argue that it is, and is actually one of the cornerstones of really making system integration easier. Why? Well, someone else posted in the comments that we should focus on other standards, like WS-Security. WS-Security is exactly the reason why we need WS-MetadataExchange. Suppose I put up a web service and lock it down. I’ve chosen to support SAML assertions embedded within WS-Security headers. I advertise this service out, but only provide a reference to the WSDL. How will any client application know that WS-Security headers are required, and that only SAML assertions should be sent? Wouldn’t it be great if a client system only needed to get the WSDL file, and then through metadata exchanges at the time the service is invoked, the client endpoint would first ask for WS-Policy information, and find out that the service is secured and requires SAML? While that may not guarantee access if the right assertions don’t exist, the systems would have at least included the proper SOAP headers rather than rejecting the request for not having enough information. This is even more important as the policies around the service change. Suppose I now need to require that some data in the message be encrypted. While the functional interface doesn’t change, the policies associated with the message interchange have. With WS-Mex, the runtime engines would pick up the new policy and make the appropriate changes without having to rebuild the Web Services client.

If we want to move toward policy-driven computing, and enable policies to be quickly enacted without requiring a code/build/deploy cycle, WS-Mex and WS-Policy are necessary.