Lloyd Hession has a simple philosophy for dealing with vulnerabilities on his company's network: Know which ones have to be fixed right away and which can be safely put off for later.
The sheer number of vulnerabilities that can exist on a network make it impossible to address all of them at the same time without serious disruption, says Hession, chief information security officer at Radianz, a provider of network connectivity services to financial firms.
So the key is to have a formal vulnerability management process to identify problems, categorize them by severity and prioritize responses, he explains.
"It's all about arriving at some sort of a risk determination and figuring how seriously you need to address it," he says. "The days of people running out and patching everything are over."
Hession isn't alone. Finding out what to protect on the network and how much protection is needed is suddenly becoming a lot more important to companies than it was even two years ago, says Scott Crawford, an analyst at Enterprise Management Associates.
The never-ending barrage of software vulnerability announcements and the constant, sometimes competing, need to fix them is pushing companies to look for more efficient ways to deal with the problem, he says.
Instead of rushing to apply costly fixes to every flaw that's announced, the goal is to take a more selective approach by prioritizing threats, adds Crawford.
"Vulnerability management tools are going to be in great demand where exposure to external risk is high," Crawford says. That's because the tools are designed to impose order on a process that has, in the past, simply been urgently reactive.
There are several components to a vulnerability management process, users say. Fundamental to the effort are vulnerability assessment scans. They help companies discover network assets and any software holes or configuration errors that might exist in them.
Vulnerability and asset classification, as well as risk metrics, are needed to help companies prioritize responses to the threats.
Mitigation and blocking measures may be needed to deal with some threats for which software updates or other fixes may not be immediately available. And monitoring and measurement processes are crucial to ensure that fixes and changes that have been made remain in place.
Detection and Remediation
A good management process helps companies identify and remediate the network vulnerabilities that really matter, says Derek Milroy, a security architect at Career Education (CEC), a US$1.73 billion company that runs post-secondary education programs.
A vulnerability management system allows companies to collect information on and understand various threats to corporate networks, and it shortens the reaction time needed to deal with them, he says. Also important, it enables IT administrators to focus their time and resources on only the problems that need fixing, Milroy says.
"It really is the core central instrumentation that enables a security function to operate within the organization," says Robert Garigue, chief information security officer at the Bank of Montreal.
Radianz has adopted several measures for managing vulnerabilities on its networks and systems. The company doesn't do too many routine vulnerability scans, Hession says. But when it does, it looks for known software holes as well as configuration errors, rogue machines and services that could be exploited, he says.
Radianz has also classified its systems into various groups depending on their importance to the organization. Critical financial and human resources systems and those belonging to senior executives, for instance, get fixed faster than those that aren't as important. Most of the company's desktops have host firewalls for detecting and blocking intrusions at the client level.
"This way, even if there are any vulnerabilities on those systems, they are not directly exploitable because of the fact that the personal firewalls are blocking it," Hession explains. "It buys you some time to go out and patch systems."
Asset and response prioritization is a key aspect of any vulnerability management strategy, Milroy says.
For the past nine months, CEC has been using an on-demand service from Qualys to perform asset discovery, asset prioritization, vulnerability assessment and analysis as well as remediation.
Like many other companies, CEC has organized its network assets into multiple security categories. It rates those categories from 1 to 5 depending on their importance to enterprise operations. Data center servers and those running crucial databases and revenue-generating applications, for instance, are considered Category 5, while some rarely used file servers might be a Category 1.
Similarly, vulnerabilities are color-coded depending on their severity, with red being the most critical. CEC runs weekly vulnerability scans of its network and prioritizes its responses based on asset importance and vulnerability severity.
A vulnerability in a database server that can be remotely exploited or for which a worm already exists might be assigned a Red 5 rating, which means that it needs to be fixed immediately, Milroy says.
In some cases, a serious vulnerability might exist in a critical system but there may be no immediate threat directed against it, in which case it may be better to do a more planned remediation rather than risk the disruption of an immediate fix, he says.
CEC largely depends on vendor classifications to determine the severity of vulnerabilities, but it also uses its own internal filters and analysis to determine whether an issue is really critical.
"I'm trying to keep it realistic. All you really care for are the Category 5 vulnerabilities," Milroy says. "Can you root the machine? Can it get hit by a worm? Is it remotely exploitable?"
Key to a good vulnerability management strategy is an understanding of the various interdependencies that exist between systems on your network, says Ed Cooper, vice president of product management at Skybox Security a vendor of risk management software.
Sometimes, for instance, fixing the problem on a single upstream server or router may be all that's needed to mitigate the risk posed by a vulnerability on multiple servers, he says.
Knowing precisely which holes to close on which server or workstation can tremendously reduce response times and help focus effort on the real threats, Cooper says.
Skybox offers a tool that allows a company to build virtual models of its entire network that it can use to simulate attacks and understand the potential consequences of vulnerabilities.
Often, the risk a vulnerability poses to a system might need to be balanced against the potential business disruption or revenue loss that might result from taking the system down to fix it, says David Giambruno, director of strategic infrastructure and security at Pitney Bowes, a US$5 billion mail and document management firm.
Software patches and mitigation approaches can sometimes interrupt needed services or functions on core systems, causing problems that ripple throughout the business.
In such cases, it's a good idea to have an "exceptions management" process under which some sort of compensating controls are put in place. It's also a good idea to make business owners aware of all potential risks and have them sign off on it, Giambruno says.
The complexity of modern networks makes it vital to have tools for automating the discovery and remediation of assets and vulnerabilities at the network, application and database levels, Giambruno says.
For example, Pitney Bowes is using a service from McAfee's Foundstone business to scan its networks for vulnerabilities once a week.
A real-time patch and configuration management tool from BigFix, helps Pitney Bowes quickly test and deploy patches across its global infrastructure in less than an hour if needed.
A database-scanning tool called AppDetective from Application Security helps Pitney Bowes scan for and discover any vulnerabilties that might exist in the database.
Mandate to Act
Vulnerability management tools and practices can provide a lot of good information about the risks companies face, but they raise their own challenges, users say.
"Vulnerability assessment gives you this view of the entire organization. Then you've got to analyze the results and ask yourself, 'What have I seen? What does it mean, and who is responsible for fixing it?' " says Garigue.
"You need to have a good quantitative understanding of what the tools are trying to tell you before you go to the business side and ask them to fix things," Garigue says. "If not, you are going to end up with a lot of cross talk."
Desktops and other client devices pose big security risks, but scanning them for vulnerabilities can be challenging because they are so portable, says Amy Hennings, assistant director of information security at George Washington University in Washington.
In the university's case, it made personal firewalls freely available to desktop users as part of a bid to improve security. Ironically, those firewalls are now making it difficult to perform vulnerability scans on the systems, Hennings says.
"The key thing to remember is that IT has limited resources," Radianz's Hession says. "So it's all about prioritizing and acknowledging that there'll always be some trade-off issues."
At the same time, though, try to keep it simple. "You don't want to make it overly complicated," Hession says.
Though companies have started adopting formal vulnerability management practices only fairly recently, there are already several tools and services available to help them through the process.
Some vendors, such as Qualys, Counterpane Internet Security and Internet Security Systems (ISS), offer vulnerability management services as part of their managed security services portfolio.
Qualys, for instance, offers an on-demand service called QualysGuard that uses a vulnerability database containing more than 4,000 unique tests to help companies identify, prioritize, fix and monitor problems on their networks, says Chief Technology Officer Gerhard Eschelbeck.
ISS offers a similar scanning service that companies can use to probe network assets such as application servers, databases, firewalls and Web server routers and switches for exploitable flaws. The service can be combined with ISS's managed intrusion prevention and managed firewall service, says Dave Ostrowski, an ISS product manager. ISS also sells a hardware appliance for vulnerability scanning.
Others, such as Foundstone which was acquired by McAfee, and nCircle Network Security, offer an appliance-based approach to vulnerability management. The Foundstone Enterprise appliance and nCircle's IP360 Vulnerability Management System are designed to let companies continuously monitor their networks and probe all discovered hosts for vulnerabilities.
An optional Threat Correlation Module allows companies to create a numerical risk ranking for each threat by tying events -- such as the emergence of exploits -- to asset and vulnerability information, says George Kurtz, a senior vice president at Foundstone.
Another vendor in this market is Skybox Security. The company sells software that a business can use to build a virtual model of its entire network, including vulnerabilities, that can then be used to simulate a variety of attack scenarios. The virtual model allows administrators to understand how systems are connected to one another in a network and to do what-if and business-impact analysis using various attack and remediation scenarios.
The goal is to give companies a "surgical list of things to do" to address network vulnerabilities in the most cost-effective fashion, says Ed Cooper, vice president of product management at Skybox.