Can an organization really cut development time more than 70 percent by embracing the agile philosophy and open architecture? The intelligence-gathering arm of the U.S. Air Force says it's done just that.
The Air Force's Distributed Common Ground System, a network of 27 surveillance and intelligence-gathering sites, projects that it will ultimately save hundreds of millions of dollars by moving to agile development, open architecture, and infrastructure-as-a-service, said Wes Haga, chief of mission applications and infrastructure programs at the Air Force Research Lab.
But the cost savings aren't the only numbers that stand out. Under the DCGS's old proprietary architecture, the IT team took an average of 84 months to bring new capabilities to the surveillance and information-analytics system, Haga said.
The team has now cut the rollout of new capabilities to under six months. "We're looking to be able to collapse it further than that," he said recently.
The move to open architecture, cloud-based IaaS, and the agile cooperative development model began in 2014, with pilot programs rolling out last year. After training more than 350 civilian workers, military members, and contractors on the new processes, DCGS is now moving to full implementation, Haga said.
DCGS, which collects data with surveillance planes and drones, has already added several new capabilities, including full-motion video capture and high-altitude exploration, that would have taken years under the previous model.
One of the biggest IT problems across the U.S. Department of Defense "is we have all these closed architectures," said Haga, one of the team members who pushed for more open systems. Workers were also stuck in "bad processes," including excessive requirements to create documents that were never read again after they were written, he said.
An IaaS foundation and agile mindset also make security testing and certification more streamlined. DCGS is now able to test proposed changes to its IT systems on its IaaS platform, and it has cut the time to get a security certification down to 30 days -- in some cases down to 10 working days -- as opposed to 18 months in the past, Haga said.
Avoiding $100 million mistakes
One of Haga's goals was to reduce the cost of decision-making for people who were making the final call on projects. In some cases, adding new functionality to the DCGS meant making "$100 million decisions" that didn't always work as intended, he said.
The cost of those old decisions was driven by the "sheer volume of processes surrounding them," Haga said. The Air Force had "these very large processes surrounding those decisions to make sure we don't make the wrong decision. Because a $100 million decision, that's kind of career killing if you make the wrong one."
But with open architecture and IaaS, the DCGS can now focus on purchasing individual components instead of an entire system, Haga said.
If adding new functionality is a $1 million decision instead of a $100 million one, "those are still large decisions, but they're not career killers," he added.
One of the core pieces of the new open infrastructure is an IaaS model, with the Air Force leading a team of multiple vendors to implement an IaaS private cloud as a platform for other technologies, Haga said. IaaS has given the organization new flexibility, Haga said.
"Tomorrow, if there’s a better widget coming out of HP, or coming out of NetApp or Brocade, we can pull that component out, because it's isolated; we can stick a different component in, and then we just map the interfaces," he said.
The move to agile methodology and an open architecture has taken work, with people at DCGS and outside contractors having to embrace a new development model, Haga said.
In the past, when the Systems Program Office started work on delivering new capabilities, "they used to throw it over the fence, throw the requirements over the fence, then the SPO would build something and then throw it back over the fence," Haga said. "Then, the customer would be upset because they didn’t get what they wanted."
But under the cooperative agile development model, the end customer is constantly involved in the process. Developers meet weekly with the group that asked for the changes to the IT systems, and once a month, the end customer spends two days working with the developers and program managers "to architect what they want," Haga said.
Part of the move to agile development is a shift in thinking that embraces change as a constant, he added.
"Under our old process … the program manager would get the requirements from the user, get ready to put something on contract, and if the customer came in and said, 'Hey, I want this instead,' then that was looked at by the program manager as a frustration," he said. "In agile, it's looked at as, 'Thank God for telling me, so I didn’t go any further.'"
Vendors are already seeing a streamlined process for adding new functionality, said Mike Lunt, vice president of engineering for Zenoss, which provides hardware monitoring services to DCGS. Under the old system of rolling out new features and services, "by the time they got the procurement, the software was obsolete," he said.