Let’s face it: network vulnerabilities are rampant, worm writers are looking for the next server application to exploit, and malicious hackers are breaching the moat and climbing up the castle’s walls. How does an organisation defend itself? It finds all the network and server holes, using a new breed of vulnerability assessment tools, and plugs them.
We recently tested vulnerability assessment appliances from Qualys and Foundstone, which take fundamentally similar approaches to finding and reporting network and host vulnerabilities. Although neither product went the extra mile and fixed our problems, we did find great benefit in what they accomplished.
The QualysGuard Enterprise Intranet Scanner appliance appealed to our minimalist sensibilities with a streamlined, small footprint appliance and clean Web site, where we configured scans and generated reports. Downsides included Qualys’ storing all your network and host vulnerability information off-site at its facilities, and requiring an annual subscription.
Foundstone’s FS1000 Appliance is a high-performance appliance focused on the mid-size to large enterprise. The FS1000 offers fast and accurate vulnerability scans with extensive and concise reporting functionality, and stores scanned vulnerability information locally. Although not as easy to manage and maintain as the Qualys solution, the FS1000 was exceptionally solid overall when it came to getting the job done.
QualysGuard Enterprise Intranet Scanner
Qualys’ appliance was straightforward to set up and get running. The QualysGuard appliance connected via the Internet and SSL to the Qualys server, which stored our scans and subsequent reports on an encrypted database at Qualys’ facilities.
We could use the Web-access capability to kick off a vulnerability scan remotely or schedule a scan for a predetermined time. The Qualys site proved extremely responsive during navigation, in some cases even more responsive than our in-house scanners. We didn’t like that we had to use file import to bring in a series of our network’s IP addresses, which we had to manually enter via cut and paste.
Each time we logged in, we were presented with a nice array of pertinent information, including our most recent vulnerabilities and our most vulnerable devices in order of priority, based on our most recent scans.
Qualys’ reporting was very thorough, and at times, verbose, to say the least. After comparing a Qualys report with Foundstone, Internet Security Systems’ Internet Scanner, Version 7.0, and the Open Source Nessus Project, Version 2.0.6, we found that Qualys’ report was overly sensitive to what we considered insignificant vulnerabilities, and it presented too much information.
Trending of historical scans was highly informative, and Qualys used well-placed graphics to illustrate important historical points. Qualys even used the most up-to-date vulnerability information to consistently reweigh our existing scans.
We liked that Qualys sent an e-mail to the manager and other assigned personnel with a password-protected hyperlink to the report every time a VA scan was completed.
Qualys is based on a hierarchical structure with a manager, scanners, and readers. We would have liked to have the ability to assign multiple managers to our Qualys account.
Although Qualys’ new remediation and trouble ticketing system is impressive, we had some issues with a single manager per organisation and the fact that Qualys didn’t integrate with Remedy, a leading helpdesk/trouble ticketing system. The company claims it can do this via its XML API.
Although we appreciated the small footprint deployment, Qualys tells us that each QualysGuard appliance can only scan about 5000 live devices per day; larger enterprises will need quite a few appliances. To its advantage, independent QualysGuard scanners aggregate into a single report.
The hardened Red Hat Linux Qualys appliance fared well when it came to scanning our network in a timely fashion. Although we could schedule Qualys scans, we couldn’t pause them and have them automatically continue at a predefined time as we could with Foundstone.
The Qualys appliance is hardened from vulnerabilities and communicates with the Qualys headquarters via port 443. Our scan and reports information is encrypted at the Qualys facility with Blowfish encryption, and is not visible to Qualys employees other than as a chunk of encrypted data. Although we have faith that Qualys would not view our extremely sensitive data under any circumstances, not having our confidential data in our immediate control did make us uncomfortable. Further, Qualys doesn’t currently support a PKI infrastructure or token authentication for data access.
Foundstone’s FS1000 appliance, which also can be purchased as a software-only solution, required a locally installed Web server for us to be able to view the reports, so we chose IIS.
The Foundstone management interface was extremely robust, boasting an extensive list of ways to run scans and generate reports. The ability to customise scans proved extremely flexible. We also liked the option of running passive or intrusive VA scans, and although this configurability was great, we found the management interface needed a little polishing and was slightly cumbersome.
Foundstone does an outstanding job in managing complexity but less than a superb job at managing simplicity. Although we could automate the vulnerability scanning of numerous subnets with beautiful trend analysis graphs, it takes up to seven clicks to run a scan for one machine.
A downside to the Foundstone management interface was that there are actually two interfaces. We used one interface on the appliance for maintenance and management, and one for generating scans, viewing reports, and remediation. We also found the responsiveness of the Foundstone interfaces a bit sluggish compared to Qualys, even though we were accessing the Qualys management interface via the Internet, whereas the Foundstone appliance was almost directly connected to our IIS Web server. We preferred to export and download Foundstone’s HTML report, which improved navigational responsiveness to a great extent.
Foundstone’s vulnerability updates were as seamless as Qualys’, which we used to update the database manually via a Windows Update-like function; we also had the appliance do it automatically.
We found reporting and trending very strong in the Foundstone product. The solution had informative yet concise reporting functionality. It also has a strong remediation component, but we did have an issue with the way that Foundstone conducts its trending analysis. We could create a trending report based on different scanning parameters, yet when we saved the reports, the software gave them the same name. We felt that trending reports should be to exact scan parameters only, otherwise the report must be renamed. This issue led to inaccurate trending reports being generated on a few occasions during our testing.
Unlike competitors, neither Qualys’ nor Foundstone’s offerings export reports in Adobe PDF format. Qualys and Foundstone can only export reports in HTML/XML. To its advantage, Foundstone includes the Foundstone Scripting Language, a scripting language for extensible vulnerability assessment.
Foundstone is focused on the larger enterprise market with its high-power dual Xenon appliance with 2GB of memory running on a hardened Windows 2000 Server, obviously geared toward scanning an extremely large number of devices (Foundstone says it has run a single appliance against up to 100,000 devices).
Foundstone is extremely fast for large networks, and was the fastest when it came to scanning our test networks over Qualys, ISS, and Nessus. Foundstone has a real luxury limousine feel to it with tons of features, rich reporting, high top speed, but a sluggish zero to 60 startup acceleration.
We were impressed that Foundstone was capable of distinguishing between obtrusive and nonobtrusive scans. While scanning, we didn’t experience any host crashes, but the option of only using a passive mode, which is the norm for Qualys, may mitigate anxiety for the overly protective network engineer.
The fact that our sensitive scan and report data is stored locally and in our immediate control also settled our nerves. Foundstone can also be customised for a PKI, and allows for different permissions for different users to view reports, allowing an even tighter integration into our enterprise security infrastructure. However, we were bothered by some of the things Foundstone did not do by default, such as using SSL to encrypt our Web sessions to the Foundstone appliance. If SSL was used, it would further increase overhead to our reporting session.
Translating vulnerabilities into a foolproof, easy-to-grasp report, one that even Dilbert’s boss can understand, was our ultimate goal during this evaluation.
Although neither Qualys nor Foundstone attained this level of perfection, they both put in valiant attempts. Both products got the VA job done with a minimum of fuss and generous functionality when it came to reporting and trending.
We saw two big differences with these products. First, Qualys stores your scan and report information at its site, a potential point of concern for some administrators, whereas Foundstone stores that same VA information locally. In the short run, Qualys also looks to be more expensive than Foundstone for the same number of scanned devices. w