Consider the following scenario: Your company has decided to invest heavily in application development, so you've spent millions of dollars on hardware and software, mounted a huge recruitment effort, and created a comfortable work environment. All the ingredients for success are in place. Yet delivery deadlines are still slipping by, despite the fact that your developers are working 80-hour weeks. Your sophisticated project planning software said this wasn't supposed to happen. Unfortunately, it has happened.

Maybe it's time to go extreme. No, extreme programming, or XP, is not about nerds bungee jumping off bridges with laptops or geeks writing code while doing back flips on their skateboards. XP is simply a software development methodology - with perhaps a bit of old-time religion and some New Age mantras thrown in for good measure.

Some of XP's main tenets are simple, commonsense ideas practised to extremes. Keep it simple. Code in small iterations and work toward fast release cycles. Design as you go - don't waste your time on a big up-front plan. Treat unit and functional testing as a crucial part of the project, not an optional add-on. Work directly with the customers and users, making them part of the programming team. Extend ownership of the project to everyone - don't let people become isolated experts on portions of the code. Refactor (that is, rewrite and improve code) constantly. And remember, it's a jungle out there, so always program in pairs.

The buddy system

Pair programming is perhaps the most controversial aspect of the XP approach. Under this method, two programmers share the same workstation to write production code. The idea is to reduce the chances of writing poor code while also fostering a spirit of collaboration.

Granted, it may seem counterintuitive to think that two programmers working together could produce as much code as they could working apart. But to XPers, the fact that more developers understand the code compensates for the slightly longer development cycles. Although individual areas of expertise may be thinner, overlapping knowledge means you'll never again be held hostage by a sociopathic, albeit talented developer who's the only one that knows how a particular part of code works.

But is it really that easy? For all its promises of better code and departmental harmony, pair programming also spotlights XP at its most dogmatic. It is true that extremely good code (no pun intended) has been written by solo developers; many creative or competent people prefer to work alone. In a sense, XP is inflexible; it does not account for different personality types and work styles, and that may alienate important members of your staff (for more on pair programming, see our test centre analysis).

Apart from changing the way developers work, XP also changes the way projects are planned. Let's face it: writing code is hard. Many software projects fail, sometimes because of bad management, sometimes because of bad coding, and sometimes because of a flawed initial concept or design.

XPers believe that if a project is doomed to fail, it's best to find out fast before you waste too much time and money. Instead of labouring over an all-encompassing design or object model and then sending your programmers off for six months (only to come back with an application that only a programmer could love), XPers code in small iterations and deliver the simplest code that could possibly work. There's no point in anticipating failures that may never arise or adding features that are never needed.

Hence, projects should be designed and developed in iterations lasting from one to three weeks. You can still sketch out long-range plans, but don't waste your time doing it in great detail. The idea is to maintain maximum flexibility in the design process. If you guess wrong in your design, you've wasted time or, even worse, locked yourself into a bad design. And even if you guess right, you could have come to the same conclusion later for the same cost. Similarly, don't design and code for reuse until it's clear that reuse is needed.

Some might argue that if your design is changing too much for an up-front specification, your project is already in trouble. But XPers claim that applications rarely look like their initial specifications, and those that do deny themselves the advantage of being flexible. If you're sailing to India, you may need to map out a general course, but you can't possibly anticipate all the changes in currents and winds along the way, so don't bother trying. In other words, don't commit until you absolutely have to.

Improving code, little by little

Even if XP goes the way of the pet rock, one idea that has already gained mainstream acceptance on its own is refactoring. Refactoring refers to a group of codified techniques for identifying and improving bad code, ruthlessly weeding out unnecessary code, and keeping the project as simple as possible.

Just how ruthless should you be? In his book, Refactoring: Improving the Design of Existing Code, Martin Fowler deconstructs one "smelly" method in one class into a total of 10 methods spread over seven classes, attempting to produce more rigorous, higher-quality, and easier-to-read code.

Even writing comments in your code, one of the first rules of Programming 101, falls under Fowler's scrutiny: code should be refactored to a state of such self-explanatory simplicity that comments are unnecessary. Although refactoring makes a lot of sense, so did comments, designing for reuse, and separate modular programming just a short time ago. It makes you wonder if we'll be reading about "defactoring" in a few years, with an example of how to combine those 10 methods back into one method.

Courage under fire

Sometimes XP takes the myopic view that everyone finds software as fascinating as developers. But many end users don't want their applications to look and work differently each time they use them, and they'd prefer not to receive frequent, small releases.

In short, for project managers, deciding to use XP on your next project may in fact feel like bungee jumping off a bridge with a laptop. You must be confident enough to surrender a good deal of control, because you'll no longer have the authority to determine what you want and when you want it. Yet the methodology is also grounded in the hard realities and pitfalls of software development, and it offers many commonsense solutions. XP may not be the final word in the evolution of software development, but it certainly is a step in the right direction.