Security giant Trend Micro is considering pulling its anti-virus software from the Virus Bulletin 100 (VB100) tests, claiming they no longer reflect contemporary malware.

In an interview with Techworld, Trend Micro's anti-malware CTO Raimund Genes was blunt about the company's feelings. "It [the VB100] was created in the 20th Century when we had one outbreak per quarter," he said. "The marketing value of the logo is high but it doesn't mean anything."

According to Genes, the VB100 pitted anti-virus products against a whitelist of viruses drawn from the collection of the independently-managed and respected Wildlist.org. However, although drawn from real-world viruses, the volume, variety and evolution of malware now rendered the use of a Wildlist subset almost meaningless.

According to Genes, other vendors were also unhappy with this aspect of the tests. He contrasted the VB100 unfavorably with the huge malware sets used by Austrian outfit AV Comparatives, which he suggested would be a better reflection of the real threats anti-malware programs were now up against.

If the pull-out comes to pass, it will be a blow not just to Virus Bulletin, but to the idea underpinning the VB100 itself. According to Virus Bulletin's website, Trend Micro has taken part in 27 tests carried out over the years by the title. The VB100 - conducted free of charge but vendors must pay to use the logo on paper marketing - would continue, but a major AV engine would no longer be there for comparison.

John Hawes of Virus Bulletin defended the VB100, saying that viewing it as a definitive malware test missed the point. It measured a basic level of competence. Vendors had to score 100 percent in order to be accredited, and had to avoid all false positives, a high but fair level of achievement against malware from a publically-known set.

"The point of the VB100 is that everyone should get everything on that list," said Hawes. "It would be a shame if they [Trend] did not take part."

He revealed that Virus Bulletin was working on a new and more comprehensive test, a sort of "VB1000", which would include leading-edge malware. He was unable to be drawn on timescales for this, but referred to the complexity of designing and carrying out such tests.

Stuart Taylor of Sophos was happy to defend the credibility of the VB100. "Because VB100 is very much a pass or fail test many people expect products to achieve a VB100 as a matter of course. Therefore it is a requirement for most vendors to participate," he said.

However "It has to be acknowledged that testing security products is becoming far more complex and testers need to address the changing threat landscape and the emerging technologies that vendors are using to counter the latest threats," he added.

A promising speck on the horizon comes in the form of the non-profit Anti-Malware Testing Standards Organization (AMTSO), a cross-vendor organization that has held a number of meetings in the hope that some new standards can be agreed on software testing. So far, nothing has come out of its initial meetings held in Bilbao and Amsterdam, but Taylor - a member of the management committee - was hopeful that a forthcoming meeting set for July, to be held at Microsoft's Redmond campus, might speed the plough.

"AMTSO is going to provide guidelines for testing which will help with testing the complexity of today's malware and different technologies. The aim is to raise the bar on testing relevance and quality," said Taylor.

The VB100 has a habit of coming up with a range of interesting tid-bits, including April's revelation that a number of anti-malware products for Vista weren't quite up to scratch.