When bad things happen to good demos

"How much money do you have in your budget? You have to be aware that this is very expensive software."

Ah, the not-so-sweet sound of the sales pitch.

Much of my work as a security manager is like that of a juggler, always keeping balls in the air. My daily goal is to deal with the next falling ball, be it a virus attack, a new e-commerce project or some suspected abuse.

This week, however, I was able to lift my gaze for a few days to think about future needs and meet with vendors. And once again, a slick product demonstration showed all too clearly one vendor's fundamental inability to understand our needs.

Two Challenges

We are meeting with vendors to address two challenges. First, we need to better manage the volume of security data we gather. Our antivirus applications, vulnerability data, intrusion-detection systems, firewalls, routers, operating systems and everything else we touch produce valuable security data, but in different formats.

It's expensive to train our staff to understand this modern-day Tower of Babel, and it takes up costly extra time when we must deal with incidents. If we could automatically translate and link security events, we would reduce costs and further improve our defenses.

The second challenge is to step up monitoring beyond our signature-based approach to detect unusual or anomalous behavior that doesn't match a known signature. Given that the SQL Slammer worm is reported to have taken less than 10 minutes to infect every vulnerable system on the planet, it's clear that waiting for an update from a vendor isn't going to work on its own.

Pulling together diverse security information is a common problem, and many vendors have products to address it, including the vendor whose salesman finished up his session with that pitch I quoted in the beginning of this article. I looked into this area a few years ago, and the products were very immature. I was hoping that the latest versions would have something interesting to offer.

We invited several vendors to demonstrate how their software could save us time, effort and cost.

The vendor I mentioned previously certainly put on a good show. Its salespeople arrived with an entire network in a suitcase and proceeded to unpack and set up servers, clients and a hub. The product ran and worked, which in some ways was the curse of their presentation. If they had stuck with PowerPoint screenshots, we wouldn't have seen what made the whole thing useless to us.

Poor Display

The tool pulled in an enormous range of data, stored it in a database, correlated root causes and generated alerts on them. It sounded good.

However, the front-end software had an awful graphical user interface. It was clunky and slow -- an unpleasant thing to force on my analysts, who would use it day in and day out.

The procedure that addressed the detection of correlated events was particularly bad. A window popped up that displayed an identification. But when I'm presented with an alert indicating that a whole series of linked things has happened, I want to see the details of all the underlying events and the reasoning used to link them, so I can understand how to respond. The sales team understood this, so they had cut and pasted the ID into a SQL report and run a report against the database.

But the whole point of this application was to reduce my team's manual work. Why couldn't we just click? Apparently, we could add our own scripts to take the action we wanted when an ID was raised.

But isn't the point of buying software to get something that does the work for me? If I wanted a framework, I'd just send all the money I have to BMC Software Inc. or Tivoli Systems Inc.

Then came the deal-killer. If you wanted to see whether new events had occurred within that correlation, the tool couldn't tell you. Instead, you had to rerun the report. So you might get an ID for a few innocuous linked events and discount it, and unless you continually reran the report to check, a bunch of horrible events could be added under your nose. My team has enough problems keeping up with the raw data without adding another layer of work.

Not to worry, though. Apparently, these "minor" issues will be resolved in the next version. If a product saves me a lot of money, then I'll pay a lot for it. It's simple, really. In this case, I don't have to worry how expensive this tool might be, since I won't be buying it.

It looks like I won't be saving money by automating our processes just yet. I'm sure some managed-service providers have ways to do this well. But since that's the core of their business, I doubt they'll sell it to me on a CD.

Perhaps the new generation of anomaly-detection software will have something worth emptying my budget into. But I had best be off to meet with more vendors before all those balls I'm juggling start coming back down.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

More about BMC Software AustraliaBMC Software AustraliaTivoli

Show Comments