Sun upgrades Internet Archive to 4.5 petabytes

The Internet Archive records a snapshot of all the information on the web

Sun Microsystems has upgraded the Internet Archive to a capacity of 4.5 petabytes. The archive has been recording a snapshot of all the information on the web every two months since 1997, storing it in a searchable online archive known as the Wayback Machine.

"The Internet Archive offers long-term digital preservation to the ephemeral internet," said Brewster Kahle, founder of the Internet Archive organisation.

"As more of the world's most valuable information moves online and data grows exponentially, the Internet Archive will serve as a living history to ensure that future generations can access and continue to preserve these important documents over time."

The entire system is installed in a single shipping container containing 63 Sun Fire x4500 servers running multi-core x86 processors. The system is run on Solaris 10 with ZFS.

"We have worked closely with the Internet Archive to ensure that the right technology platform is in place to handle and manage growing amounts of the world's most valuable data, and that it lives on for future generations," said Dave Douglas, chief sustainability officer and senior vice president of cloud computing at Sun.

The Internet Archive is run from a site in the Presidion complex in San Francisco, and currently holds around three petabytes of data. This increases by around 100 terabytes each time the web is mirrored.

The information was held in an offsite datacentre, and Sun will now host the container on its Santa Clara campus.