Challenge of Data Services
I was recently having a conversation regarding data services and my recent posts (here and here) on horizontal versus vertical thinking put it in a new light. My experience has shown me that data services are one of the most controversial topics within an enterprise.
Nearly all solutions, somewhere along the line, are going to need to access some form of structural data, typically from one or more relational databases. The debate around data services begins with a discussion on whether or not all access to the data should go through a service or not. I’m not going to get into that part of the debate in this post. The point of this discussion lies more with the organizational challenge of creating a data services layer.
One reason (but certainly not the only one) that is sometimes used to justify a data services layer is to maintain control over how the databases are used. All it takes is one bad SQL statement to cause a significant impact to all systems using that database. When organizations see this problem, the instinct is to centralize the development of data services. By creating a team with expertise in database access technologies, the assumption is that you’ll get high quality services that will prevent system failures due to data unavailability. The challenge with this approach is that this assigns ownership and service boundaries along a technical domain: data access. Per my previous postings, assigning ownership based on technical boundaries implies a horizontal domain. Horizontal domains are ones where the capabilities are becoming commodities and can be done in a standard fashion. Service teams in a horizontal domain typically strive to minimize the number of services involved. This is where the challenge occurs. Data access is anything but a commodity. While the techniques used to access data may be standardized (SQL), the structure of data and the combinations of data desired by consumers is certainly not.
In his book Human Interactions,
Keith Harrison-Broninski pointed out that everyone deals with information differently. We file it differently, we take notes differently. Trying to standardize information access is inherently problematic. Why is this? Because information retrieval operates in a vertical domain. We’re retrieving information for the purpose of doing a particular task. So, we have a services team that is viewing the world in a horizontal fashion, with their primary goal being to minimize the number of services they provide. On the other side of the equation, we have vertical thinking consumers that want data for a particular purpose in a format that is optimized for what they need to do. Clearly, if the consumers have the most influence, the goal of the service team to minimize the number of services is not going to happen. If the service team has the most influence, the goal of the consumers to have optimized services is not going to happen. We simply have a situation where the two (or more) teams involved have competing goals. You can’t say that either one is right or wrong, only that they are competing with each other. There can be sound justification on both sides, so the organization must mediate and try to create a win-win situation.
I think this subject is a great example of the concepts of horizontal and vertical thinking. Things can go very smoothly when both the consumer and provider are consistent with their understanding of a particular domain as either horizontal or vertical. When there is disagreement, and one group sees things as horizontal while another sees things as vertical, the relationship is going to be problematic. If you can make it a win-win situation, that is the best scenario. If one party or the other simply has a view that is inconsistent with the view of enterprise architecture, that’s a communications and education issue for EA. EA needs to set the policies for what constitutes good architecture, and if a particular functional domain is deemed to be horizontal (or vertical) that information should be well communicated so the individual solution architect approaches its usage properly.
One Response to “Challenge of Data Services”
Leave a Reply
I am of the opinion that the top level decomposition in SOA is only to services – no databases. In this manner, nothing from any other service can access data in a given service’s database(s). Especially when considering event-driven interactions, there will be few cases where this will even come up.
I’m squarely in the camp of services being responsible for both data and the logic around how that data is managed. Thus, I consider data services (those without connection to business-level data aggregation) to go against SOA.
Inside a service, there is definitely a case for consistent data access technologies and patterns, but those would be modeled and implemented with components, not services.
I find these clear boundaries as to what a service’s properties are to make it much clearer how to actually go about building Service-Oriented Enterprises.