Software tackles 'stupid' forum postings

A new open source application is being developed to cut stupid or abusive comments on internet forums.

The StupidFilter Project aims to make its software available for free by the end of the year, and is planning a Firefox application by 2008.

The project is run by systems administrator Gabriel Ortiz as part of an initiative to take back internet conversation from "those who seek to debase it ".

"Too long have we suffered in silence under the tyranny of idiocy. In the beginning, the internet was a place where one could communicate intelligently with similarly erudite people," he says on the StupidFilter website. "It is time to fight back."

The software will sit on the server side of a website and examine new postings for signs that the contributor is not making sense. These might include repeat letters (e.g. 'amazinggggg') and text-speak such as 'LOL' (for 'laughs out loud').

If the software feels that the post is unsuitable it will send it back to the author and suggest that it needs to be rewritten.

Ortiz maintains that this kind of filtering is a lot tougher than spam filtering because of the intricacies of the language. He also expects people to try and defeat the system in the short term.

"We certainly hope they will because that implies they are no longer generating text statistically likely to be stupid," he said.

"It is true that an obvious attack on the StupidFilter would be to salt a short, stupid comment with a long excerpt pasted from, say, Project Gutenberg. But we think it is reasonable to count on the laziness of the stupidest commenters not to do this."

The project has taken over 200,000 user comments from sites such as YouTube and will be using them to develop and test the software.

The team is also recruiting internet users to help improve the filters, although such moderators have to pass a grammar test and submit a selection essay that weeds out over 90 per cent of applicants.