Microsoft bristles over failed antivirus test

Microsoft has fired back at a security software testing firm that failed two of its antivirus suites.

AV-Test recently gave Microsoft's Security Essentials and Forefront Endpoint Protection programs low scores for their antivirus protection potential. Microsoft has now fired back at the firm's claims by decrying their method for testing.

"It is [difficult] for independent anti-malware testing organisations to devise tests that are consistent with the real-world conditions that customers live in," wrote programme manager for Microsoft's Malware Protection Center, Joe Blackbird, in a blog post.

Blackbird argued that AV-Test failed Microsoft because its testing practices are not in line with the real-world usages of consumers.

Redmond uses an antivirus detection setup that prioritises malware prevalence and potential customer impact in its software. While AV-Test uses a testing program that doesn't prioritise threats.

According to Microsoft, its method allows for a much faster turnaround for finding the greatest possible malware threats. Meanwhile, AV-Test's methods would still give credit for detecting viruses that have little effect in the real world, it claimed.

"Microsoft brings up a few good points in their blog and we've been discussing these items with Microsoft and many other anti-malware vendors already," AV-Test chief executive Andreas Marx told V3.

"These issues are not specific to testing but describe the general problem in determining prevalence and the impact on the entire user base. Depending on what you count, you will get different results."

According to Marx, AV-Test uses a family-based approach to testing that allows for the firm to find potential virus issues more generally. Marx says that the methodology allows the company to discover malware that can affect a high number of individuals.

"As of today, every two seconds we see three new malware samples, which are summing up to a few million samples per month. Instead of looking at millions of samples, our focus is on the unique families," continued Marx.

"Out of every family, we select recent samples in order to use them in our tests. So the impact of these samples is indeed low, however, the impact of the malware family is considerably high. We favour the family-based approach over the sample-based one because of today's malware situation."

This isn't the first time antivirus testing methods have sparked debate. In 2008, a group of security vendors released a set of testing guidelines for antivirus software. The group released the guidelines because it felt that many security testers where unfairly favoring certain types of security software detection methods.