Todd Biske: Outside the Box

Archive for the ‘REST’ Category

I’ve been reviewing the FHIR (Fast Healthcare Interoperability Resources, http://www.hl7.org/fhir) specification and they have an interesting concept called a compartment.  Per the spec:

Each resource may belong to one or more logical compartments. A compartment is a logical grouping of resources which share a common property. Compartments have two principal roles:

• Function as an access mechanism for finding a set of related resources quickly
• Provide a definitional basis for applying access control to resources quickly

Let’s look at these statements one at a time. First, the component concept provides an access mechanism for finding related resources. One very common compartment in the specification is Patient. Other resources, like Condition, clearly have a relationship with Patient. So, if I want to find all conditions that a particular patient has, I actually have two paths for doing this.

• GET /Patient/[id]/Condition
• GET /Condition/?patient=[id]

[id] is the unique identifier in question. In this case, both of these requests should return the same thing.  But it’s not quite that simple.  Take another resource, Communication, which deals with secure messages sent as part of patient care.  In this case, we have:

• GET /Patient/[id]/Communication
• GET /Communication/?subject=[id]
• GET /Communication/?sender=[id]
• GET /Communication/?recipient=[id]

The first example returns any communication that involves the identified patient, whether to, from, or about.  The Communication specific inquiries only allow for inquiry by the attribute of the resource where a Patient identifier can be specified.  It just so happens that in the earlier case, the relationship within Condition is represented in a patient attribute.

Independent of whether you think this is a good or bad thing, this approach where there are two ways of getting to the same resources creates a decision point for the organization.  In a large enterprise, it’s entirely possible that the implementation for different resources may be handled by different teams.  With two (or more) different ways of doing this, it creates the risk of two (or more) different implementations.  It also creates a situation where a resource that can be a compartment needs to make sure that any time a new related resource is defined and implemented, they also need to make a modification to provide the compartment-based inquiry.  Once again, if this is a separate team, this means coordination. Anyone who’s worked in an enterprise knows that the more teams that get involved, the more challenging it becomes.  

These are not insurmountable difficulties by any stretch of the imagination.  In the case of the implementation, the compartment resource should simply act like a façade and make the appropriate calls to the resource (i.e. the implementation of the first URL in the examples above simply turns around and makes the call(s) below them to complete the inquiry, such as Patient calling Condition, or Patient calling Communication).  In the case of the coordination, that’s a matter of education and oversight to make sure it happens.  The greater risk is probably that too many things get defined as a sub-structure within the compartment resource, rather than defined as standalone resources.  This can be avoided by recognizing when a proposed resource has multiple compartments.  Take the following requests:

• GET /Practitioner/[id]/Condition
• GET /Condition?asserter=[id]

These inquiry would give me a collection of all conditions that a particular practitioner has ever dealt with.  If Condition  wasn’t a standalone resource, and instead a sub-structure within Patient, how would I go about forming this query?  It can be done, but it’s probably not going to look as simple as what is shown above. This is where I see the hidden strength of this compartment concept.  By recognizing where we can have multiple ways of organizing a particular collection of data and traversing relationships, we can then make good design decisions on what our resources should be.

Finally, FHIR also mentions that the compartment concept can also play a role in access control.  I haven’t dug into this one as much, but I think it may have some potential. The challenge lies with data that really has multiple owners.  As a patient, I may want to use an OAuth model to grant access to my health records to a mobile app I’ve downloaded. My doctor may want to do the same thing for an application he or she uses as part of my care.  The compartment approach could give independent access paths for each of these channels with their own policies.  Again, I need to give this one more thought, but I can definitely understand why HL7 put the bullet point about access control in their specification.

What are your thoughts about this notion of compartments?  Good thing? Bad thing?  Have you implemented a similar approach? What were the pros and cons of it?  Let’s start the discussion.

I had an interesting conversation with some colleagues around resource design that I thought would be helpful to share. 

The starting point was a simple question:

Should price generation be a HTTP POST or HTTP GET?

There’s solid reasoning for either of them.  Let’s start with HTTP GET.

From a consumer’s perspective, a GET probably seems very intuitive.  For how most people think about prices, it’s static information, so why wouldn’t it just be an attribute included on a GET of some other resource, like a Product, right? 

For the purposes of this conversation, however, price is something that is computed at the time of the request.  In other words, some supporting static information exists (like list price), but the actual “price” charged to the customer is dependent on other contextual parameters.  Cases where this exists are the end price you pay at Amazon after taking into account shipping preferences, account status (e.g. Prime member?) or the price you pay when you buy a car.  These prices are determined on the fly and may only be good for a limited amount of time, because the contextual information is subject to change. Hopefully, you can also see that “price” is actually a complicated piece of information.

Is HTTP POST beginning to sound better? Where this fits very well is that the “Price” is really a custom resource generated for that particular context.  The customer, even though they may use the phrase “get me a price,” is really saying, “generate me a quote.” At this point, we’re creating a new resource.  

But there’s one more thing.  What if price calculation is expensive? If I make this a POST and generate this every time, won’t my costs go through the roof? Well, they don’t have to. There’s no reason that a subsequent POST with the same data can’t return a resource from cache  in this scenario.  In reality, you are probably updating the expiration date of the resource, so POST still makes sense. Furthermore, if you provide a unique ID for the calculated price resource, HTTP GET can be used to retrieve it again, it just shouldn’t update the expiration policy.

So, out of this, I came up with the following guiding principles on deciding whether calculated/derived data should be its own resource:

• Can the data stand on its own? That’s always a question for any resource.
• Does the calculation require contextual data from the consumer to perform the calculation, or are all the parameters already part of the potential parent resource?
• Is there value in keeping the calculated data around for some time to avoid re-calculation?

Hopefully these guiding principles will help you out.  If you have other suggestions on factors that help this design decision, please feel free to share in comments or via your own blog post.

I had a brief conversation with Nick Gall (Twitter: ironick) of Gartner on Twitter regarding designing for change. Back in the early days of SOA, I’m pretty sure that I first heard the phrase, “we need to build things to change” from a Gartner analyst, although I don’t recall which one. Since that time, there’s been a lot of discussion on the subject of designing/building for change, usually tied to a discussion on REST versus WS-*. Yesterday, I stepped back from the debate and thought, “Can we ever design for change, and is that really the right problem?”

As I told Nick, technology and design choices can certain constrain the flexibility that you have. Think about the office building that many of us work in. There was a time when they weren’t big farms of cubicle and they actually had real walls and doors. Did this design work? Yes. Was it flexible enough to meet the needs of an expanding work force? No. I couldn’t easily and quickly create new conference rooms, change the size of spaces, etc. Did it meet all possible changes the company would go through? No. Did the planners ever think that every cubicle would consume the amount of electricity they do today? What about wiring for the Internet? Sometimes those buildings need to be renovated or even bulldozed. The same thing is true on the technology side. We made some design decisions that worked and were flexibility, yet not flexible enough for the change that could not have been easily predicted in most companies, such as the advent of the internet.

Maybe I’m getting wiser as I go through more of these technology changes, but for me, the fundamental problem is not the technology selection. Yes, poor design and technology selection can be limiting, but I think the bigger problem is that we have poor processes for determining what changes are definitely coming, what changes might be coming, and how and when to incorporate those changes into what IT does, despite the available predictions from the various analysts. Instead, we have a reactive, project-driven approach without any sort of portfolio planning and management expertise. To this, I’m reminded of a thought I had while sitting in a Gartner talk on application and project portfolio management a year or two ago. If I’m sitting in a similar session on service portfolio management 5 years from now, we’ve missed the boat and we still don’t get it. Develop a process for change, and it well help you make good, timely design choices. The process for change involves sound portfolio management and rationalization processes.

I wanted to call attention to four good podcasts that I listened to recently. The first is from IT Conversations and the Interviews with Innovators series hosted by Jon Udell. In this one, he speaks with Raymond Yee of UC Berkeley, discussing mashups. I especially liked to discussion about public events, and getting feeds from the local YMCA. I always wind up putting in all my kids games into iCal from their various sports teams, it would be great if I could simply subscribe from somewhere on the internet. Jon himself called out the emphasis on this in the podcast in his own blog.

The next two are both from Dana Gardner’s Briefings Direct series. The first was a panel discussion from his aptly-renamed Analyst’s Insight series (it used to be SOA Insights when I was able to participate, but even then, the topics were starting to go beyond SOA), that discussed the recent posts regarding SOA and WOA. It was an interesting listen, but I have to admit, for the first half of the conversation, I was reminded of my last post. Throughout the discussion, they kept implying that SOA was equivalent to adopting SOAP and WS-*, and then using that angle to compare it to “WOA” which they implied was the least common denominator of HTTP, along with either POX or REST. Many people have picked up on one comment which I believe was from Phil Wainewright, who said, “WOA is SOA that works.” Once again, I don’t think this was a fair characterization. First off, if we look at a company that is leveraging a SaaS provider like Salesforce.com, Salesforce.com is, at best, a service provider within their SOA. If the company is simply using the web-based front end, then Salesforce.com isn’t even a service provider in their SOA, it’s an application provider. Now, you can certainly argue that services from Amazon and Google are service providers, and that there’s some decent examples of small companies successfully leveraging these services, we’re still a far cry away from having an enterprise SOA that works, whichever technology you look at. So, I was a bit disappointed in this part of the discussion. The second half of the discussion got into the whole Microhoo arena, which wound up being much more interesting, in my opinion.

The second one from Dana was a sponsored podcast from HP, with Dana discussing their ISSM (Information Security Service Management) approach with Tari Schreider. The really interesting thing in this one was to hear about his concept of the 5 P’s, which was very familiar to me, because the first three were People, Policies, and Process (read this and this). The remaining two P’s were Products and Proof. I’ve stated that products are used to support the process, if needed, typically making it more efficient. Proof was a good addition, which is basically saying that you need a feedback loop to make sure everything is doing what you intended it to. I’ll have to keep this in mind in my future discussions.

The last one is again from IT Conversations, this time from the O’Reilly Open Source Conference Series. It is a “conversation” between Eben Moglen and Tim O’Reilly. If nothing else, it was entertaining, but I have to admit, I was left thinking, “What a jerk.” Now clearly, Eben isn’t a very smart individual, but just as he said that Richard Stallman would have come across as to ideological, he did the exact same thing. When asked to give specific recommendations on what to do, Eben didn’t provide any decent answer, instead he said, “Here’s your answer: you’ve got another 10 years to figure it out.”

I’ve been meaning to call out a blog from Anne Thomas Manes posted back in March, along with a message in the Yahoo SOA group, as they were finally something that, in my opinion, added some useful information to the ever present REST versus SOAP/WSDL debate. Normally, I stay out of this religious war when it kicks up in the blogs or the Yahoo SOA group, but that’s not to say that I don’t care about it. The fact is, I’m a practicing enterprise architecture in a Fortune 500 company, so I need to be providing guidance when a team comes to me asking about REST versus SOAP/WSDL.

In most of the conversations about REST and SOAP/WSDL, it’s usually a comparison of a single SOAP endpoint (a single URI) to a single REST endpoint (again, a single URI). Invariably, the conversation always wind up being about the uniform interface (GET/PUT/POST/DELETE when using HTTP) versus the non-uniform interface (whatever operations are defined in WSDL) tunneled through the transport (POST when using HTTP) in SOAP. I’ve always felt that this was a bit of an apples and oranges debate because the REST endpoint is exposing a resource, and the SOAP endpoint was exposing a service. When thinking of services in a conceptual, technology independent manner, the mapping to a resource just didn’t seem as straightforward.

The comment that Anne made that helped put things in the right perspective was this:

Service consumers interact with the service through the set of resources it exposes. In other worlds, the resource model is the interface to the service. Each resource exposes a uniform interface (e.g., GET, PUT, POST, and DELETE), but an individual resource is not the complete service.

This made it very clear. If you’re trying to go from a conceptual service model to a design based on REST, an individual service does not equate to a single REST endpoint. Rather, it equates to a collection of REST endpoints that together comprise the resource model associated with that service. In my opinion, the lack of an understanding around this concept is probably also why most of the “REST” services out there really aren’t REST, but rather are XML over HTTP without the WSDL and SOAP envelope. The people involved are still trying to do a single endpoint comparison and not thinking about the resource model as a whole.

Now, I’ll admit that this insight still doesn’t solve the uniform versus non-uniform debate, but I do think that it brings us to the point where a valid comparison of approaches for a particular problem could be taken on within an organization. Thanks Anne!

James Governor of RedMonk has a post that he claims is the “Most exciting idea in ages: an ATOM/LDAP mashup.“ It’s actually a very interesting idea to me, especially because he suggests applying it to the management domain. I’ve previously posted (here and here) about the convergence of metadata, and how I’ve seen parallels between Service Repository, Configuration Management databases, Asset Management systems, and potentially even LDAP. So, if we’re doing an ATOM/LDAP mashup, is ATOM equally applicable to other items in the metadata management domain. I suspect that it will be. Nearly all management specs I’ve seen have a resource-oriented view, and it would seem that the combination of ATOM and REST could be a very good fit on top of this. Hopefully James will keep us all updated on the progress of this exciting idea.

Some of the blogs I follow have been simply giddy about the recent statements on REST from Anne Thomas Manes of the Burton Group. About the only thing I agree with on some of the comments is that Anne is extremely smart. Beyond that, this is a case of people seeing what they want to see. Of course, some will accuse me of doing the same, but that’s okay. All of our blogs are simply forums for stating our own opinions, so that’s what it should be!

First, let’s call out where the debate exists. The debate has been around REST versus WS-*, not REST vs. SOA or ROA (Resource Oriented Architecture) vs. SOA. Even there, you could narrow it even more, which was eloquently captured by Erik Johnson in a comment on Stefan Tilkov’s blog (which came to my attention courtesy of Don Box’s blog).

“It seems to me that people attracted to REST (in whatever form) are rebelling against interface-based programming more than WS-* itself — at least that’s my excuse.”

There have been endless debates in the blogosphere and mailing lists over whether or the message payload is part of the interface or not. Clearly, REST is all about having a uniform interface, and if you include message semantics in that interface, it seems that it would be a difficult thing to achieve. The uniform interface is GET, PUT, POST, and DELETE. There’s been some recent discussion about media types and their role in implying message semantics, however, the WS-* proponents will argue that there’s a huge gap between application/xml and having XML Schema (courtesy of WSDL).

Anyway, back to the statements from Burton Group. While most of the bloggers are commenting on this news story at SearchWebServices.com, I first caught wind of this directly from the source via the Application Platform Strategies Blog at Burton Group. My take on this is that Anne first recognizes that REST isn’t going away anytime soon, as is increasing in mindshare. No arguments here, and there are plenty of REST bloggers out there popping champagne bottles over this statement.

The piece that I feel is getting lost in some of this discussion is that, as Anne points out in the APS blog:

REST is not the same as “plain old XML” (POX). POX refers to the format of a message payload, and it says nothing about architectural style. It just says that you don’t wrap an XML message with an XML envelope (e.g., SOAP or Atom). More to the point, not all POX applications are RESTful, and not all RESTful applications are POX.

This is probably the biggest myth right now. There are lots of people that are using XML over HTTP, thinking that they are using REST, when in fact, they aren’t. Again, Anne nails it with this statement:

REST is not simply technology–it’s an architectural style that’s fundamentally different from they way most developers design systems today. It requires a noun-oriented approach to designing systems rather than one based on verbs. I know quite a few people that have been studying REST for years who still struggle with RESTful design practices. Understanding the basics of the style is easy. Truly groking it and being able to apply it to real-world situations is much harder.

This is where I’d like to see some constructive conversations. When we’re talking about SOA, we’re talking about services. Services represent capabilities. Capabilities are typically associated with verbs. REST is resource-driven. To me, that would mean that I should apply it where I need a resource oriented architecture, not a service oriented architecture. The question is where is this appropriate? Clearly, there are many calls that do nothing more than retrieve data, and it would seem that a REST-based approach for this would work very well. The question, however, is whether a resource oriented view is sufficient.

One of the things I like about SOA is that it can help establish better lines of ownership, which theoretically, can allow IT to operate more efficiently. Because these are based on services, however, they are more likely to be aligned along functional boundaries rather than on resource boundaries. Resources are shared across functional boundaries, so if my unit of composition is at a resource level, how do I deal with these concerns? I’m not suggesting that it can’t be done, but I think this is where there is lots of room to grow, as Anne points out. In the SearchWebServices example, Anne provided a light bulb example, which (pun intended) should turn on the light bulb for people on what they’re getting into:

“A REST application to turn on and off the lights in your building will require you to design a URI for every light bulb and then you send it on/off messages,” she explained. “It’s not like I have a single service that manages all my light bulbs. It’s a very different approach to designing a system. And it’s going to be really hard for developers to get their hands around it.”

So what is my opinion on all of this? If it’s doesn’t come across from my blog, I tend to very pragmatic. I think there are places for both WS-* and REST, and that will continue for the foreseeable future. REST makes lots of sense to me when we’re dealing with browser-based clients (e.g. AJAX). WS-* makes lots of sense to me for service-to-service interactions. I do fall more on the side of formal interfaces, and as a result, I want to learn more about WADL. I’ve yet to see a solution that is doing REST by the book, most examples I’ve been fall more into the POX/HTTP category, or using an HTTP GET with query parameters to return data as XML (all read only). That doesn’t mean they don’t exist, I just haven’t seen them. In any case, debates such as these keep things interesting. There’s always risks that it will strictly be a battle of religions rooted in opinions (which never get resolved). Involvement of people like Anne Thomas Manes and others that fall into the pragmatic group in the middle can ensure that the debate progresses to appropriate application of these approaches to where we leverage the strengths of both, and minimize their use in areas where the weaknesses are exposed.