A suggestion that data mining technologies similar to those used to detect credit card fraud can also help in the battle against terrorism has received a less than enthusiastic reception from US government officials as they work to establish a Department of Homeland Security, according to an IT specialist with the Council on Foreign Relations.
James Shinn, a fellow at the Council on Foreign Relations and a lecturer at New Jersey's Princeton University, provided details Monday about a report called "Red-Teaming the Data Gap," which he and colleague Jan Lodal presented to government officials in January. He was speaking at the Industry Advisory Council's Executive Leadership Conference here.
The report lays out a case for establishing a counterterrorism system that uses data mining to look for suspicious patterns within data contained not only in federal government databases, but also state, local authority and commercial databases such as those held by car rental agencies, Shinn said.
If data that government and private organizations held prior to Sept. 11 had been subjected to what Shinn described as "fairly trivial pattern checking," the hijackers would have been caught, he said.
One conclusion of Shinn's "Red-Teaming the Data Gap" report was that the U.S. has a competitive advantage in its superior technology to detect and deter terrorists.
"The cost of a terrorist strike is so catastrophic that we can't afford to deal with them in the forensic, after-the-fact approach," Shinn said. "It's pretty important to deter them, hence the IT approach."
Shinn said the report included a "blue team recommendation" encompassing a long-term systems re-engineering effort costing billions of dollars and taking years. In addition, the report suggested a "red team recommendation" designed to give the government an interim solution.
The "red team recommendation" involved using commercial off-the-shelf software and "a small group of good engineers from the agencies involved, and you slice through databases opportunistically. Do it fast; do it cheap; and do it quick," he said.
The cold shoulder that the "red team recommendation" received was attributed to "a lot of legitimate privacy concerns," Shinn said. He also cited the fact that federal agencies typically don't share information, and are using "creaky, stovepipe legacy systems." But he predicted that when the next attack occurs a lot of carefully debated privacy protections will be tossed aside.
"I think we would be far wiser to tackle them up front rather than be driven by an emergency," he said.
Despite the lack of uptake of the report's recommendations, Shinn said, he and his team received numerous e-mails from vendors showing how their tools might fit into a data mining system and wondering why they didn't get more attention from the government.
In addition, he said, some government officials who recognize the need for sharing data are pushing ahead with their own data-sharing objectives, at the risk of being reprimanded for doing so.