Protocol abuse targets vulnerabilities in many types of devices and applications, from firewalls, VOIP controllers and VPN gateways to intrusion-prevention systems and other perimeter defense. Despite the considerable investments made in security infrastructure, many vulnerabilities remain undetected.
To alleviate protocol abuse, a new class of product -- the security analyzer -- can help IT departments assess the security of IP-based products, service or applications. A security analyzer utilizes a rigorous process, complete with an audit trail and remediation scripts, to find and fix vulnerabilities before deploying systems and software into production networks.
A security analyzer connects to a system and emulates hacking by generalizing techniques hackers employ and applying these as a comprehensive set of protocol attack vectors in a systematic, repeatable fashion. Unlike source code analyzers and vulnerability assessment tools, security analyzers can be used by nonexperts to assess systems and applications in a lab environment.
Security analyzers detect known and unknown zero-day vulnerabilities by subjecting the target system or software to many permutations and combinations of protocol abuse attacks. To analyze for unknown vulnerabilities, maximum protocol abuse is achieved through extremes of valid, invalid or unexpected inputs that violate the protocol's specifications. Examples of these extremes include formatting a field's type, length or value incorrectly, inserting illegal characters and adding trailing blanks.
The key to finding protocol vulnerabilities is understanding a protocol's potential weak spots. Comprehensive coverage is critical because, just as the failure of a single part can cause an airplane to crash, a single protocol vulnerability can expose an entire network to attack. But to be truly effective, security analyzers must also operate efficiently with a finite and well-conceived set of protocol attack vectors.
A security analyzer subjects the target system or application to a large number of attacks -- potentially millions. During this onslaught, the state of the target is continuously monitored. Details about any anomaly or unexpected result are logged in a database that provides a complete audit trail to establish baselines and historical regressions that are useful when comparing products, releases or configurations. An analyzer also can create a self-extracting Linux-based executable file capable of replicating the exact attack for each vulnerability. This file then can be shared with the vendor or development team to expedite the remediation effort.
When the target under analysis fails or locks up (the intended result of many hacker attacks), the security analyzer issues a reset command through an out-of-band channel. If this fails, the analyzer reboots the target system by cycling its power off and on again. Such automated controls allow the full security analysis to be completed while unattended, potentially overnight.
The ability to pinpoint vulnerabilities in a stand-alone system or application provides a practical way to compare competitive product offerings, possibly against a benchmark, before making a purchase decision. Additional post-purchase applications include alerting the vendor to a vulnerability and assisting with the remediation effort, verifying patches or profiling new releases as part of a change management process, and evaluating and contrasting specific system configurations. An analyzer also can assess the effect of changes in the enterprise security policy, evaluate internally developed software for vulnerabilities, and perform complete security audits.
Security analyzers will enable IT departments to minimize vulnerabilities -- and their costly consequences -- throughout enterprise networks without increasing the budget for defense-in-depth protections or security consulting.
Guruswamy is co-founder and CTO of Mu Security. He can be reached at firstname.lastname@example.org.