Intrusion detection systems (IDS) set off so many false alarms they can cause more trouble than they are worth, according to chair of Australia's Information Security Interest Group (ISIG) Mark Ames.

Responding to the results of the first ever IDS product comparison conducted "in the wild" --which found the systems couldn't even be relied upon to detect intrusions -- Ames said they are certainly not a magic bullet.

"Almost all IDS products in default setting generate so many false alarms they are useless; the trick is to use them wisely if an organisation is to get any value out of them," he said.

"The systems they are supposed to help protect have to be analysed to determine what exactly the IDS should be detecting; this is hard work and requires detailed risk analysis.

"You won't find out how to do that in the IDS manuals and it certainly doesn't come out of the box."

Ames was responding to product testing undertaken by Computerworld's sister publication in the US Network World which found the detection systems are "guaranteed to detect and consume all your available bandwidth" but little else.

Of the seven IDS products tested, several crashed repeatedly under the burden of false alarms and when real attacks came along, some products didn't catch them and others buried the reports so deep in false alarms that they were easy to miss.

Testing also found overly complex interfaces made tuning out false alarms a challenge.

Ames said there's little point in implementing an IDS if there is no response plan in place. Systems have to be reasonably secure or alarms will go off all the time, he said.

An IT security manager, who wished to remain anonymous, said most IDS systems have a built- in expert engine designed to help filter and correlate logged events but an experienced security analyst is necessary to interpret output.

"After being flooded by hundreds of alerts the average IT manager will start to ignore them creating a dangerous situation," he said.

The eight products tested -- from Cisco Systems, Intrusion Inc, Lancope, Network Flight Recorder Security (NFR), Nokia (running on OEM version of Internet Security Systems RealSecure 6.5), OneSecure, Recourse Technologies and the open-source Snort package -- all ask too much of their users in terms of time and expertise to be described as security must-haves, Networld World testers said.