When ZapThink talks about Service-Oriented Architecture (SOA), we generally try to avoid semantic arguments about what is and what isn’t SOA, but rather try to focus on specific characteristics that identify Service-oriented systems and the benefits they provide organizations that adopt the architectural approach. In particular, we like to focus on three core tenets: loose coupling of Service consumers and providers, composability at a variety of levels of granularity, and event-driven, asynchronous styles of interaction that allow for a wide range of usage scenarios.
While these characteristics might seem distinct, they are in fact related to each other. The ability to compose arbitrary Services in environments of long-running transactions requires a certain measure of loose coupling. However, architects seem to grapple with the term “loose coupling”. Some erroneously think that systems are either loosely coupled or tightly coupled as if the determination of coupling is a binary evaluation. The reality is that the concept of coupling, like that of granularity, is a relative term. Something is more loosely coupled than something else if it allows for a greater degree of variability and freedom for that particular aspect. Simply put, coupling is a measure of dependency and commonality required by a Service consumer and provider to be able to achieve a result.
In that light, loose coupling is a spectrum. A system can be tightly coupled in one aspect while being loosely coupled in another. Many architects say that they want complete decoupling of systems so that any variation in the system can be handled without having to reimplement any of the Service consumers or providers. As we’ve discussed in past ZapFlashes, complete decoupling is incredibly difficult and expensive, if not impossible. So, what architects should be aiming for is achieving the right level of loose coupling to facilitate business agility without imposing huge costs.
As such, what ZapThink has seen is that there are really seven levels, or perhaps aspects, of loose coupling that architects should consider in their SOA initiatives. The more degrees and levels of loose coupling they add to their SOA efforts, the greater those systems can deal with change.
Loose Coupling the Implementation
One of the most fundamental promises of SOA is that Service consumers are blind to the implementation technology used by the Service providers, and vice-versa. Indeed, the whole idea of the Service abstraction is that technology implementations are hidden. This most basic of SOA characteristics is realized using standards-based, interoperable specifications or protocols that provide a contract-based interaction between Service consumers and providers. Web Services certainly fits the bill as does REST and other styles of Service implementation. Indeed, it’s not the choice of interoperable specification (whether XML based or not) that enables this level of loose coupling, but rather the Service contract. A properly written Service contract will allow Service consumers to bind to a wide range of Service provider implementations without have to know at all whether the Service implementation is done in Java, .NET, PHP, C++, or Basic. All SOA implementations must be at least loosely coupled at the implementation level, but clearly this is not enough to guarantee the sort of complete loose coupling so desired by the business.
Loose Coupling the Service Contract
XML, Web Services, REST, and other specifications hide the implementation of the Service such that when the Service implementation changes, the Service consumer does not need to be the wiser about it. However, what happens when the Service contract itself changes? A simple change to acceptable inputs or functional behavior of the system can have profound impact on Service consumers. As such, architects that want to progress their SOA initiatives beyond simply Web Service integration need to address Service contract change management in such a way that contract changes don’t cause Service consumer breakage.
This might seem to be a difficult proposition, but there are plenty of best practices, architectural approaches, and technologies to facilitate this level of loose coupling. First, a proper Service contract that is interface-neutral will alleviate much of the problem of Service contract change management. The Service Description Framework (SDF) shared by ZapThink in our Licensed ZapThink Architect (LZA) boot camps is one of those technology-neutral Service contract templates. In those approaches, new Service contracts don’t simply replace old ones. Old contracts are never deprecated unless a transition path is provided for Service consumers. This can be as simple as a transformation or as complex as a Service-oriented process that is triggered when Service consumers request old (and deprecated) versions of the Service contract. As such, contract change must be handled as a matter of policy. Who gets what version of a Service should be controlled via metadata, not programmatically or via configuration of tightly-coupled infrastructure.
Active management and Service intermediaries further simplify the Service contract change management issue by providing exception management and handling of Service contract differences. And of course, SOA Governance plays a huge role here in helping to manage the sort of change and versioning issues that arise. But probably the best practice in this arena is the use of late-binding approaches that leverage run-time registries and contract-based binding to abstract the binding specifics of Service consumers and keep Service contract changes from propagating. The topic of late binding is itself a deep conversation that requires rethinking the way that Service consumers bind to Service providers, and one too long for this ZapFlash.
Loose Coupling the Service Policy
So, putting into play Service contracts that abstract implementations and late-binding, intermediary-enabled, registry-based systems that allow for Service contract change without breakage enables a great degree of variability in the system, but we’re still far from done. Indeed, even if the Service contract stays stable, a small change to a Service policy can have tremendous repercussions, as discussed in our Butterfly Effect ZapFlash.
Companies looking to address Service variability should handle changes to policy the same way they handle changes to Service contract: late-binding, Service intermediary-enabled, registry-based, and governed. A policy is a form of metadata, as are contracts, and in fact, the only difference between a Service policy and a contract is that a policy can apply to any number of Services. Because policies control many aspects of the non-functional parts of a Service, architects need to include in their architectural plans methods and practices for dealing with Service policy versioning and deprecation as well as policy versioning that leverages the technologies and practices established for Service contracts. Companies need to test policies just as they test Service implementations, and manage policies as rigorously as they manage Services. Doing so not only makes the system as a whole more reliable, but enables one more level of loose coupling.
Loose Coupling the Process
Of course, loosely coupling the Service consumer from a Service provider only provides agility if the Services aren’t composed together. Once you have a business process that composes a bunch of Services together, we have another area of potential tight coupling. What happens when the process changes? Ideally, a Service consumer should not have to know at all when a process is reconfigured. Fortunately, achieving that level of loose coupling is fairly straightforward.
The movement in the SOA and BPM markets has been towards separating the process definition layer from the underlying implementation. By defining processes in metadata and exposing those processes using Service contracts (the recursive vision of process compose of Service and exposed as Service) abstracts the implementation of the process from the Service consumer. In fact, in such a scenario, a business process is really a form of Service implementation. And just as Service contracts provide loose coupling of Service implementations, they provide loose coupling of Service-oriented processes.
Loose Coupling the Data Schema
If we’ve enabled all the loose coupling levels defined above, companies should be able to make arbitrary changes to their Service implementations, contracts, policies, and processes without breaking a thing (or a sweat). Yet, that’s not sufficient to handle the changing needs of the business. What happens when the underlying data schema changes? If a Service consumer and provider need to have a common understanding about that in order to have a conversation, we have tight coupling as defined above. Therefore, organizations need to further their loose coupling goals by enabling dynamic and heterogeneous change to the data schema shared between Service consumers and providers.
To some, this might seem to be a very complicated task. Computers aren’t people, after all, and so they can’t process any changes to the format or definition of data. Yet, not all hope is lost here. First, Services aren’t really interoperable unless they can understand and process data in common. This means that while there might be semantic dependencies between them (which we will address shortly), there does not need to be structural data dependencies between them. Schemas are key to Service data interoperability. However, what happens when Schemas change? One key approach is to address information and schema management in the same way you approach Service metadata management. Data schema can be treated as a form of metadata and the use of exception management, transformations, Service intermediaries, and Data Services (described in more detail here) are all key to enabling loose coupling of data structure. Implementing a Data Services layer introduces loose coupling in a way that provides a separation of concerns, so that underlying data infrastructure changes are insulated from the business Services above.
Furthermore, companies need to align the efforts of those maintaining the data schema with those maintaining the Service metadata. Why is it that the folks who maintain the schema are not part of the architectural team? Schema are no different than Service contracts. They represent a form of metadata that encodes business requirements. As such, we frequently admonish companies to bring their data and information architect teams into their enterprise architect fold. The simple act of this alignment and the establishment of metadata-enabled change management will allow for loose coupling of data schema and structure.
Loose Coupling the Infrastructure
Before we try to slay the last dragon of loose coupling, it’s important to note that all these levels of loose coupling doesn’t matter a whit if it all depends on a single vendor’s implementation. Too many times we interact with architects who claim that their systems are loosely coupled, but if they move their implementation from one Enterprise Service Bus (ESB) or Service infrastructure to another then all hell will break loose. How can they truly say their implementations are loosely coupled with such a huge dependency in mind? Enterprise architects worth a grain of salt will demand that their Service implementations be infrastructure neutral. That means that at any time, the company can change their infrastructure without having to rebuild all the Service consumers and providers.
Many vendors promise this sort of interchangeability, but few deliver. In fact, the industry as a whole seems to be moving towards single-vendor SOA platforms that will keep many SOA initiatives tightly coupled at this layer. This ZapFlash serves as warning to firms that want to achieve high degrees of loose coupling: keep your implementation infrastructure neutral and you will be successful. Otherwise, your SOA initiatives will only enable partial business agility.
Loose Coupling at the Semantic Layer
The final level of loose coupling is the most challenging of all. Even if a company is enabled to make all the changes it wants to Service implementation, contract, policy, process, infrastructure, and data structure, problems still arise whenever there is change to semantics. As we detailed in our Semantic Integration: Loosely Coupling the Meaning of Data ZapFlash, if we impose the data structures of Service providers on Service requesters, the result is every bit as tightly coupled as previous architectural approaches. In order to provide the promise of seamless data integration, we must transcend simply loosely coupling the application interface and in addition provide loose coupling at the semantic level.
As we detailed those ZapFlashes, in order to achieve the sort of semantic data integration we are seeking, we must implement dynamic service definitions. In essence, the definition of the Service interface must change based on the context of the Service requester. As a result, a Service can change its contract between invocations. For example, the fact that a Service provider requires first names to be no longer than 40 characters should not require the requester to know that fact. The contracted Service interface is supposed to provide that isolation. Service interfaces must therefore become much smarter. Instead of having to know ahead of time what specific data requirements are needed by a Service, the Service requester should be able to dynamically discover a Service interface that can not only provide the needed functionality, but also understand the information payload.
The ZapThink Take
SOA as a whole is an exercise in loosening the coupling of heterogeneous systems that have to cooperate to meet the needs of the business. As such, as companies seek to mature their SOA initiatives, they should be seeking to iteratively loosen the coupling of their systems at the various levels described above. The most amazing part of this loose coupling exercise is that the greater the degree of variability that can be tolerated by the system, the greater agility that you’ve enabled for the business. Even more so, if companies can manage to address all seven layers of loose coupling while maintaining a flat cost of architecture, then we’ve reached the desired state of the IT-Business relationship: IT can implement every requirement and desired outcome expressed by the business without imposing any additional time or cost impedance.
To be specific, in a system that is enabled with all seven layers of loose coupling, changes to Service implementations, business processes, Service contracts, Service policies, runtime infrastructure, data schema, and semantics won’t break the system. Can you think of any requirement by business not addressed in those levels of loose coupling? This vision of agility is precisely the aim of SOA, and it clearly requires much broader understanding of SOA than simply loose coupling of the implementation, which is the only capability provided by Web Services.