Lack of storage keeps secrets of the universe hidden

Studies into the building blocks of the universe are being constrained by the amount of data that can be captured and stored, according to a scientist at the Cern laboratory in Geneva.

David Foster, head of communications and networks at Cern, said that attempts to unlock the secrets of the universe by studying events at the particle level involve huge amounts of data being discarded before it can be analysed.

"There are several million readout channels from a detector. If you were to digitise all the information, you would be collecting data at about one petabyte a second," he said.

"There is no way that the technology of today can record any of that, so we have to filter it down."

Foster explained that the data is filtered to try and determine whether the events in a particular interaction are interesting or not.

"There are about 20 particle interactions every 25 nanoseconds, so you get an idea of the volume of the data and how rapid the decision making must be as to whether this is an interesting event or not," he said.

"You want to get around to a data rate of something like a few hundred megabytes per second."

Cern is trying to establish the existence of new particles by accelerating them to enormous speeds in a 27-kilometre facility before forcing them to collide and studying the result.

Foster pointed out that this kind of research has always needed large amounts of computing power.

"We estimate that the analysis of this information will take 100,000 of today's fastest computers," he said.

Foster added that the research facility maintained its computing needs using the principle of "buy 'em cheap and stack 'em high".

He estimated that 15 petabytes of data are analysed by Cern's research community every year.