Making false positives go away

  • Andre Yee (Computerworld)
  • 28 January, 2004 09:49

Network intrusion-detection systems (IDS) have long been recognized as a necessary component of a multilayered security architecture. False positives have long been the bane of IDSs, leading some to even question the value of these products. In the past, many security professionals have found the effectiveness of intrusion detection limited by the need to process a large number of alert notifications, many of which are either incorrect or irrelevant.

However, new network IDS products are appearing that help to tackle the false-positive problem with a smarter detection engine that uses three key technologies: operating system fingerprinting, alert-flood suppression and meta-alert correlation.

Passive OS fingerprinting

For many users, a false positive occurs when they are notified in a nonvulnerable scenario. These scenarios exist because most network IDSs don't take the host vulnerability profile into account when detecting for attacks. For example, a Windows remote procedure call attack will be flagged as an alert even if there are only Linux machines on the network. The key to reducing false positives in this scenario is to provide context-based alerting, in which host information is incorporated into the detection framework. New IDS products use passive operating system fingerprinting to build host profile information. Operating system fingerprinting determines the target host operating system based on matching IP and TCP header parameters with those of known systems. A database of host profile information is constructed and used in the detection engine.

When an "attack" packet is identified, the IDS will perform a cross-reference against the targeted host information to determine if the target host is vulnerable. If the target host is deemed "not vulnerable," the IDS may log the notification for forensic purposes but would visually suppress the alert. The result is that the security administrator only deals with "potent" alerts for which the host is actually vulnerable. Some vendors have extended the concept of basic operating system fingerprinting to application fingerprinting to even greater effect. As an example, one could choose to visually suppress Microsoft's Internet Information Server attacks targeted to an Apache Web server farm, hence cutting out unnecessary alert notifications.

Suppressing a flood of alerts

Another big complaint about false positives occurs during what's known as an "alert-saturation" scenario. This can happen when a virus or worm is widely propagated throughout the network. Examples include the MS Blaster or SQL Slammer outbreaks that occurred over the past year. What happens is that the network IDS will repeatedly notify of the same attack, causing a flood of alerts. From our experience, it's not inconceivable that many older IDSs will flag thousands of alerts in less than 15 minutes of monitoring.

To address this concern, products are beginning to incorporate a technology component known as alert-flood suppression. Incorporating rules or parameters into the IDS sensor allows for smarter processing in an alert-saturation scenario. These next-generation sensors will recognize such a scenario and suppress repeated identical alerts. They preprocess "potential alerts" prior to notification on the basis of rules using parameters that take into account alert type, source IP address, destination IP address and time window.

For instance, a smart sensor may be configured to ignore repeated alerts of the same kind that are targeted to the same host and occur within a 30-second window. Often, the IDS may regard such alerts as identical and suppress repeated notification. The IDS may, however, keep a count of the identical alerts for statistical analysis.

Meta-alert correlation

Meta alerts are generated by the correlation of two or more alerts, possibly from different sensors. Meta-alert correlation rules are defined in the IDS by the security administrator to enable the generation of a higher-priority alert whenever certain conditions related to lower-level alerts are fulfilled. As an example, a security administrator might want to be notified whenever host-scan activity coming from the same source IP address escalates over a 10-minute window and moves across the network to other servers. Scans coming from the same source IP address with increasing rate may indicate escalating activity with greater penetration and hence would necessitate a higher-priority alert.

If meta-alert correlation wasn't enabled, each of these host scans might be viewed as discrete activities and possibly dismissed as insignificant.

While this example illustrates that meta-alert correlation can result in earlier attack notification, it isn't always obvious how this technology impacts the reduction of false positives. Security administrators' complaints about false positives often result from having to process low-priority alerts that turn out to be generally benign. It's incumbent upon the security administrator to identify from these low-priority alerts any patterns or conditions that indicate a more urgent scenario. By setting meta-alert correlation rules, the security administrator can primarily focus on high-priority meta-alerts instead of parsing through hundreds of lower-level alerts that are of little consequence.

Meta-alert correlation parameters will typically include time window, event count, event type, IP address, port number and event sequence.


The next generation of IDS products will be more about intrusion defense than intrusion detection alone. These systems must have the capacity to prevent intrusions based on precise trusted detection. To deliver on this promise, current-generation products must address the issue of false positives by building a "smarter" detection framework. This detection framework should provide the following:

  • Enterprise context for detection analysis
  • "Smart sensors" that alert on parameterized rules
  • Correlation and aggregation of alerts

IDS vendors are incorporating technology advancements such as operating system application fingerprinting, alert-flood suppression and meta-alert correlation engine. These advancements begin a new phase of trusted intrusion-defense products that will deliver enterprise value to the security organizations.

- Andre Yee is president and CEO of NFR Security, a developer of information security products in the US.