As I write this, there is a barely audible, irregular thumping noise emanating from a dusty 386 PC in the corner of the room. I fear a system failure is on the way.
The noise reminds me of a recent echo cardiograph I undertook in which the full extent of my advanced addiction to caffeine was transformed into sound. Thumping sounds caused by palpitations emanating from the most important muscle I possess. I no longer drink coffee by the way.
If the noises from the PC in the corner get any louder, I can think of a number of people who work with me who may find themselves experiencing stress - rather then caffeine induced - palpitations. You see, that dusty PC in the corner is running a system last touched about seven years ago. I kid you not. Seven years ago. It has given tireless, 24/7 service during those years with only occasional weeding, mulching and the odd tender loving warm boot from tender, loving hands.
Sure we have backups. Weekly backups. We even have a machine in hot backup mode ready to take its place if it goes belly up. Despite all this, we are still nervous. Nervous because the system has not been touched in years. Nervous that a number of its hardware and software components are now antiques in IT terms. Nervous that while some of the system was built in-house, some of it was outsourced and the people responsible for its development have long since disappeared to find themselves in the jungles of South America or snowboarding their way around the USA.
We are nervous because we know in our hearts and souls, that under that irregular thumping noise, lies a ball of tightly integrated software applications.
For 'tightly integrated' in the above sentence, think of a ball of mud. Unless you are extremely lucky, you have some balls of mud in your sphere of influence too. Can you think of any systems you know that you are afraid to touch? Systems that have become so tightly knotted over the years that no one human brain understands it all both at a conceptual and detail level? I thought so.
Now. Given that balls of mud are the harsh reality of many IT systems and given that tight integration was the catalyst for creating them, isn't it odd that the phrase 'tightly integrated' still sounds so appealing? I have seen many a C-level executive nod approvingly as the phrase 'tightly integrated' drops from the lips of the system vendor or the system integrator. Tight is good.
Actually, tight is almost always bad.
To most business people, 'integration' means process integration. That is, some applications are tightly integrated if they work as one to perform some business function. It is a very unfortunate fact of life that the quickest and cheapest way to get started with process integration is by means of spot-welding systems together. This so called point-to-point integration accounts for most of the world's "tight integration" code developed by system integrators.
The downside of tight integration can be severe. First, the costs of modifying the point-to-point integrations can be significant. With the inevitability of change, this is more than just of mere academic interest. Indeed, I have known systems where the point-to-point integration code had to be redone *from scratch* after an innocuous-sounding change request from the business managers.
Second, and perhaps most significant in these days of high application counts, tight integration does not scale. By that I mean, as the number of point-to-point integrations grows, the number of "spot welds" increases exponentially. I'll leave it to your powers of deduction to conclude what happens to the dollar cost!
So, tight integration sounds good but burns a hole in your wallet. What is the alternative? Here things get interesting. The buzz phrase is 'loose coupling' achieved by means of middleware.
Loose coupling comes in a number of flavors - some of which do not address the fundamental problems of tight integration at all.
For my money, the most useless form of loose coupling is to allow parts of an application to run on machine A with parts of it running on machine B. This really only gains you the ability to deploy your tightly integrated applications across a bunch of loosely coupled wires. It is still tightly integrated in terms of the domino effect of changes. Furthermore, unless it is approached very carefully it does not significantly reduce the scaling problems either. DCOM anyone? CORBA anyone?
For my money, the most powerful form of loose coupling is temporal decoupling. With this approach, two systems A and B send/receive instructions/data to each other over a reliable messaging infrastructure in which there is no assumption that any request will receive an instantaneous answer. The reliable messaging infrastructure guarantees you that an answer to any request will be provided - but does not tell you when.
It has been my experience that if you build systems this way, true decoupling can result. By true decoupling I mean the ability to plug-replace any component piece without bringing the whole applecart crashing down. If my dusty 386 in the corner were running a truly temporally decoupled collection of applications, it would not be a health risk to me or my staff. We would sleep easy at night because we would know - on a component by component basis - what its interfaces were to the other components. We would also know that should one component fail, the reliable messaging plumbing interlinking the components would allow us to replace it without losing any partially completed business processes.
So, the next time a software or integration vendor intones the phrase 'tightly integrated' you might ask what that really means - especially in terms of ability to change and evolve the tightly integrated system. Also, the next time you hear the phrase 'loosely coupled' you might ask what that really means. If the answer is RMI or RPC Web Services allowing pieces of the system to be geographically separated - yet temporally coupled - I suggest you dig deeper with your questioning to see what the proposed loose coupling approach actually gains you.
Loose coupling is good for you, but only if it goes beyond the knee jerk middleware layer approach that simply distributes the tight integration over loosely coupled wires. Unfortunately, at the moment, there is a lot of this going on in the form of RPC style Web Services. The good news is that the advantages of temporal decoupling - and the reliable messaging infrastructure that typically goes with it - are coming to the fore.
I suspect that temporal decoupling will form one of the key differentiators of Service Oriented Architectures (SOA) from previous approaches to middleware in the years ahead. That will not stop sales guys pitching tightly integrated SOA architectures to you though. Old habits die hard.