Yahoo launches Mail Visualisation to track spam

Yahoo has launched a project which the company hopes will provide a clearer picture of how email traffic travels on its networks.

The company said the Mail Visualization Project will provide users with a world map view of where traffic is going and how spam messages are being intercepted and blocked.

Using a real-time analysis tool based on the Hadoop platform, the visualisation tool is able to provide a live report on which regions are currently accounting for the highest volume of network usage.

"Through the visualisation project, we can demonstrate the technology behind the Yahoo Network and show the impact that the tech has on you, the consumers of Yahoo’s great technology," wrote Yahoo Labs engineer Markus Weimer and grid architect Andreas Neumann in a blog post outlining the project.

"Our technology analyses all this anonymous data with the help of Hadoop and identifies spam patterns so we can then use really intelligent algorithms to predict future email patterns that will differentiate 'good' and 'bad' senders."

Though recent reports suggest that spam traffic has begun to tail off slightly as a result of law enforcement activities and botnet shutdowns, Yahoo reports that it still blocks four out of every five messages on its mail service as spam.

The company said that it is able to reduce spam reports by as much as 65 per cent by using the Hadoop platform.

Hadoop has seen its popularity grow in recent years and many vendors and enterprises have looked to integrate the platform for large-scale database and big data collection and analysis. Recently, Oracle made the Hadoop platform a key component in its much-publicised Big Data Appliance rollout.