The software testing industry needs a reality check in order to thrive in a landscape that’s vastly different to the one it faced when it emerged as a discipline, according to the global head of QA for equities at JP Morgan, Brett Rogers.
Rogers, who is Sydney-based but heads a transnational team with a headcount of 250, said that the QA industry needs to come to grips with the new world of software – methodologies, tools and application architecture have changed dramatically over the last two and a half decades.
“QA evolved a lot from the manufacturing industry,” Rogers told the iqnite conference in Sydney today.
“If we look at what manufacturing did – how they process things, how they did quality control... manufacturing is where we got a lot of our processes from in QA. So manufacturing at the time, and even now, [is] very procedural-driven, very process-driven.
“For example, let’s think of a production line manufacturing a hammer. You don’t change every day, or every minute, how you produce a hammer. It stays the same, I’d guess, for probably three years maybe. Then somebody decides to put a different coloured handle on or maybe some more holes in the grip, but producing a hammer doesn’t change...
“[P]robably 25 years ago, software was not the dominant part of technology; it was all about hardware. So hardware did control software. It is now the exact opposite. Hardware is a complete commodity. It runs off the shelf...
“We’ve changed the industry 180 degrees. It’s now completely different – what we need to do and what we need to check. So as such, a lot of those ideas we had 25 years ago, which we’ve probably maintained even to today, are now outdated.”
While manufacturing frequently relies on sample-based quality assurance, this practice is not applicable to software.
Software QA managers and their teams need to face up to the failure of automation, Rogers said. Not that automation itself has failed, but its use for software testing is still too partial, and too many times teams are relying on manual testing.
“I struggle to think of any industry who has promised so much and failed to deliver,” Rogers said.
“We have been less than successful [when it comes to automation], and we are all responsible for that,” Rogers said, adding that on average only some 15 per cent of test cases are being automated.
The second places Rogers sees QA struggling is with what he dubbed the “race to zero”: “Trying to get everything to cost nothing.”
“How do we do that?” Rogers said. “'Well, let’s outsource everything right? Then it’s not on my books; it’s not my problem; someone else can look after it.'”
While lowering headcount through outsourcing or offshoring is warmly received by financial analysts, it can have a deleterious impact on QA because of ‘context leakage’.
“Context is actually key to a lot of things in life, especially testing and QA reporting,” Rogers said.
“So context leakage happens over time; it happens naturally. It also happens, like Chinese Whispers, as you pass things on. The further away you are from the source of information, the more context is lost. As you outsource anything, you move a team offshore, whatever you do, there is context leakage the more hands, or the more heads or voices, that information needs to go through.“
Within 10 years, and possibly as soon as three to five years in many cases, Rogers is predicting 50 to 70 per cent fewer software QA testers.
The future role of QA lies in being able manage coverage, end-to-end testing, which is important given the rise of service-oriented architecture, and being able to articulate risk to the business.
“Someone executing a thousand test cases, what the hell does that mean?” Rogers said. “I have no idea. I want to know: What’s the risk? What have I covered and what haven’t I covered?... I don’t give a stuff about how many test cases. I’d be happy to have one test case, if we covered everything in the system.”
In the future, QA “will still be responsible for quality, but managing quality. Managing the coverage and articulating the risk. That’s key to how we succeed in the future,” Rogers said.