There’s no such thing as bug-free software. According to most studies, the average number of software bugs per 1000 lines of code hovers between five and 20.

Most are errors in syntax that never surface as problems. But with applications ballooning to millions of lines of code, the chance of a show-stopping mistake affecting any given application rises precipitously.

Examples are easy to find. Take Microsoft’s ongoing server vulnerabilities, Oracle’s first release of its E-Business Suite 11i, or Netgear’s router firmware released last May that continuously pinged the University of Wisconsin’s public Network Time Protocol servers, resulting in an inadvertent DoS (denial of service) attack. The perception among many IT customers is that bugginess has reached crisis proportions.

The effect of bugs on productivity is high. The Sustainable Computing Consortium, a collaboration of major corporate IT users, academics and government agencies, has estimated that defective software cost businesses around the globe $US175 billion in 2001. The National Institute of Standards and Technology in 2002 released a report stating that software errors cost the US economy $US59.5 billion per year. The study found more than a third of that expense could be eliminated by improved testing that enabled earlier and more effective identification and removal of defects. To tackle all manner of software quality problems, enterprises are establishing best practices in the development phase, using third-party testing software to catch errors, and hiring third parties to inspect code after the fact.

Stopping bugs before they breed

According to Jeff Payne, president and CEO of code-assessment services provider Cigital, software failure occurs for three reasons.

“First, software is probably the most complex [thing] we try to build today,” Payne says. Second, the nature of software is such that no foolproof set of rules can be created that will absolutely eliminate bugs. The third reason, Payne says, is “the fact that developers and people who build software just do a very poor job of testing, validating, and building what they’re doing”.

Most analysts agree that although a separate QA procedure should always be in place, the best way to increase software quality is to have developers test as they go — and to establish procedures that ensure business-side requirements are well-understood. Bugs often spring from common human error, but of equal culpability are poorly conceived or poorly conveyed business requirements. When something doesn’t work as intended, users don’t care whether the cause was a programmer’s slip of the finger or a misread requirements document. “Best practice is to build quality in. Don’t try to test it in,” Payne advises. For Cigital, proper software engineering means specifying what is to be built, and then architecting and designing before coding and testing. Using test-driven development, code is tested early in the process, rather than waiting to test the entire system when it’s more expensive to fix problems. In addition, software quality reviews and artifact analyses help companies that build software cut costs by eliminating expensive human hours for reworking and late lifecycle testing costs.

“You cannot catch all of the bugs through QA,” says Alberto Savoia, co-founder and CTO of startup Agitar, developer of Agitator bug-testing software (due for release in early 2004). Savoia also advocates getting developers more involved in bug detection rather than leaving this function to QA personnel.

“Really, the issue of software as a whole is essentially [that] software is still handmade. It’s developers getting together and still hammering it out by hand,” says ZapThink senior analyst Jason Bloomberg. He advocates XP (extreme programming) and “agile” software methodologies that “more tightly link developers to the users who will use the final product”.

Agile methodologies are specifically intended to ensure software meets business-side requirements, especially when requirements are changing, Bloomberg says. But the practice loses effectiveness when scaled beyond small project teams. “Often, the project is too large to have a small team of developers with some users on it,” he says. The requirements are too numerous and the repeated evaluation of applications by the business side becomes too heavy a burden.

Besides, many developers are naturally resistant to feedback. “To a large degree, developers still see themselves as artists,” says Alexander Falk, president and CEO of Altova, an XML tools developer. He stresses that software development should be more like engineering and less like art so that developers can be open to different approaches.

Management must be more attuned to software quality issues, says Jeff Klagenberg, director of product management at Reasoning, a code inspection service. “When you get to business management, there’s often a disconnect with the software development side and [the fact] that services and tools exist out there to make it easy to remove these defects,” Klagenberg says.

Apparently, word is getting out. Reasoning revenues have increased 50 per cent or more each quarter this year, according to the company, and the number of lines of code inspected has increased more than 200 per cent per quarter. The company’s prices start at 18 cents per line of Java, C, or C++ code examined in a process that mixes manual and automated techniques.

The business risks of lax inspection can be high. Through code assessments, Cigital customer MasterCard has uncovered security issues in applications, according to Simon Pugh, vice president of infrastructure and standards at MasterCard. “Certainly, as a result of their services, we have found and prevented a number of problems that otherwise would occur,” such as flaws in software that could have been exploited by a hacker, Pugh says. For example, in a smartcard application developed by a third-party company and subsequently analysed by Cigital, the application contained a back door that would have allowed a rogue Web site to interact with a card and obtain a PIN number, he says.

Empirix, which provides load and performance testing, has found code problems such as e-commerce site users’ pages getting transposed so each received the other’s personal information, says Colin Mason, performance consultant in the Empirix hosted testing service group.