The 'Cinderella' syndrome hits testing during the development process says Marshall Hodgekiss, director of independent commercial and accounting software developer Bear Mountain Systems. He believes testing is of critical importance in the development process, but says that, unfortunately in his 25 years experience as a software developer, he has seen relegated to the "Cinderella of the development process", along with developers' internal documentation.
"It is expected that it will be done but it is not given the required resources, recognition, time, or priority," Hodgekiss said. "In larger development teams where there are separate testing resources, unrealistic schedules or a lack of priority often cause product to be prematurely handed off 'for testing' resulting in excessive development-testing iterations."
Hodgekiss said that in the end this practice does more damage to the project schedule than if a higher emphasis had been given to developer-level testing in the first place. Moreover, in smaller development teams the required testing may not occur at all for the same reasons.
"A frightening number of projects reach alpha, beta and even user release without adequate testing; the end user ends up being the tester."
Hodgekiss recommends development managers "make it plain" that testing is a priority and push for the necessary schedules and resources to allow it to occur.
"The modularity of the product will also have a significant bearing on testing," he said. "Failure to modularize the product with defined, documented interfaces will complicate testing and open the project to breakage when another issue is corrected."
Automated testing products have the potential to assist in the testing process, Hodgekiss said, but adds there are numerous issues associated with their use.
"The creation of testing scripts requires resources, perhaps more than manual testing, but the payoff is in later retests," he said.
"Test scripts, manual or automated, must not just test whether the correct behaviour occurs for correct input, but correct handling of invalid inputs.
"In a real sense testing must try and break the program not just confirm that it behaves to spec on correct inputs."
This 'testing for extremities' mantra comes in handy because "after all, the users have not read the specs and they are very inventive when it comes to doing the wrong thing".
Agitar Software has made a niche for itself in the code testing space and just this year opened a local office in Sydney. Asia-Pacific vice president Jeff Pope said the company's products are about software development quality enabling customers to "develop faster and get the bugs out while they build code", a concept claimed to be new.
"Instead of debugging software after it has been developed, Agitar's position is to get the bugs out of the code while you are building it," Pope said. "And we provide metrics people can use to measure the quality they are getting."
Pope said this is most apparent with outsourced development to different companies.
"People with time to market issues are very conscious of the whole quality issue [and] improving your release cycle is not about just throwing people at development," he said. "If there is an outfit that does a lot of internal development or outsources or offshores we bring lots of value to them because we can monitor the quality of code their partner would give them back."
Pope recommends not just trusting a code review but to explore some metrics and measure what's been developed as the development process proceeds.
"A lot of people set out with the right intentions [and] they know they have to follow processes, but the whole time-to-market thing cuts in and organizations are squeezed to deliver faster so they cut the window down," he said, adding that Agitar's testing software works the same whether you are in the business of selling software or developing it for in-house use. "And [when companies] cut corners they are not doing as much testing as they ought to."
Pope said in the past developers haven't been able to build the code and automatically test it as they build. Proper testing during development reduces the amount of QA (quality assurance) work, but does not eliminate it entirely.
Borland's regional product director for Asia-Pacific, Malcolm Groves, said that over the past five to six years attitudes towards testing have changed.
"A few years ago testing was seen as something done at the end of the development cycle," Groves said, adding that with the rise of extreme programming and more agile development, testing is becoming more entrenched into development processes.
"QA isn't only responsible for an application - the whole process is - and testing is still seen as a stand-alone process," he said. "A big issue, particularly at the immature stage of the development cycle, is the people think the tool is the answer. Tools will help speed up testing but won't deliver good processes."
Groves said projects need to be tested at most stages for a range of different things like usability, performance and load handling, and regression.
"A big part of testing is what you are doing with the results. Who is it visible to? It needs to be visible to everybody," he said. "We find that the best practices for QA is to look at the whole process."
Depending how much the software is worth to your business, Groves said the amount spent on testing will vary appropriately but a QA budget of 30 percent is "a good measure".
"Testing is not a phase at the end of a project so the money should be spent on the cycle," he said. "It's disappointing that during the rush to finish a project it's the QA stage that gets chopped."
Borland Australia and New Zealand managing director Peter McAlpine said the company has been advocating an "intelligent collaboration lifecycle".
"If you can catch defects early in the development cycle the impact becomes less expensive," he said, adding that it can be upwards of 40 times cheaper to deal with problems during development than after a project is complete.
"The more money spent upfront the better the outcome downstream," he said.