It's an old joke that IT development projects are on track until a week before the deadline, at which point the customer discovers that the project is really six months behind schedule. The developers may have worked long hours, but it often turns out that nobody fully understands the status of their project deliverables.
One reason the status and deliverables are unknown is that the developers' processes are hidden' from the project manager and end users; it's almost as though the developers were coding in the dark. I learnt the significance of this on a recent trip to India, which is rapidly becoming recognised as the centre of software development excellence. Of the 10 software organisations in the world that had achieved a Level 5 rating on the Software Engineering Institute's (SEI) Capability Maturity Model scale last September, five were Indian. And an amazing 40 per cent of the world's 59 high-maturity organisations (those at Level 4 or 5) are in India.
The scale provides the basic explanation of the "hidden process" dilemma. An organisation at Level 1 has no defined process; hence, neither the developers nor the manager nor the customer knows what processes are being performed. In contrast, an organisation at Level 3 or higher does have a defined process, and if anyone wants to see it, there's a document that describes it in detail. Good project management goes hand in hand with formal, disciplined processes that are carried out consistently; everyone knows what deliverables are associated with each activity and there should be very few surprises along the way.
But what about end users who have been burnt on previous projects and who aren't passionate believers in SEI? They might ask: "How do I know that your developers are following a formal process?" If the project manager says: "Here's a description of the process, and it's the same one we followed on our last 10 projects", the cynical user might still say, "We don't care about your last 10 projects! How do we know you're following it this time?"
There are two familiar ways to answer this question: produce frequent prototypes throughout the project, and produce voluminous progress reports, updated weekly. But in today's Internet time' environment, that may not be enough. A mission-critical e-business project may last only four weeks. If it takes a full week to find out that you're building the wrong system, you've already lost 25 per cent of your time.
An alternative, as some of the best Indian software companies are demonstrating, is to make the software process completely visible so that the customer doesn't have to wait passively for a week before the project team delivers a status report. If the team is using a formal process that's implemented in a Web-enabled, workflow-automation mechanism, if the team performs all its work within that environment, and if the automated process is instrumented' with appropriate metrics, then end users can literally watch the process in motion, in real time, through their own Web browsers. They can observe requirements being created, modified and approved; they can observe design reviews, with metrics about the number of defects that were identified; and they can track defects to see whether any festering problems threaten the project's success.
Why is such an approach coming from India? Simple: most of the large Indian software organisations are doing outsourcing projects for European and American clients. But even with their impressive SEI certifications, Indian companies face a credibility problem: their clients just aren't sure if they can trust a project team that's located thousands of kilometres away. The Web-enabled visible process' approach goes a long way toward eliminating this problem and also provides a vast improvement in the project's manageability. If IT organisations in the rest of the world don't start adopting some of these advanced methods and processes, they may find themselves at a severe competitive disadvantage.