When President Clinton leaves the White House in January 2001, his legacy will include an e-mail system with some 40 million messages in it. Those records, by law, must go to the National Archives and Records Administration (NARA).
The NARA has the task of storing and preserving those messages for as long as it has to, which is pretty much forever. But "for practical purposes, there is no durable digital media," said Kenneth Thibodeau, director of the electronic records program at the NARA.
And that's not the only problem. The volume of records is increasing dramatically as agencies begin turning over records generated on PCs in a variety of formats. In a recent report to Congress, the US General Accounting Office said the National Archives faces a "substantial challenge."
A Super Solution
The NARA thinks that the only way it may be able to keep up with the electronic-records boom -- from tens of thousands to millions annually -- is to replace its homegrown PC system with supercomputers.
It's an effort that may ultimately produce ideas that benefit businesses with long-term records-preservation needs, said users and analysts.
"It certainly is an issue that we're facing and are going to face a lot more as we automate a lot of our health records," said Dave Bowlan, manager of information management at Kaiser Foundation Health Plan in Pasadena, California.
Kaiser is developing a strategy for long-term management of electronic records and will be watching the NARA to see if it can come up with an affordable solution, Bowlan said.
The NARA has been working with the San Diego Supercomputer Centre (SDSC) to attempt to resolve the storage issues. Using 1 million e-mail messages as a test, SDSC has "shown us ... that you can handle this" with massively parallel processing, Thibodeau said.
SDSC is using a supercomputer to speed migrations of data to the new medium, reformat data to meet new standards and import metadata into new catalogs, among other things.
SDSC has also been using Extensible Markup Language tags to keep track of all the documents -- something that "has a lot of market support," Thibodeau said.
Long-term storage is another problem because there's no storage medium that can guarantee multidecade life. But moving data to a new medium, as the NARA does every 10 years, actually has benefits. Each new storage medium typically holds more data at a lower cost and also offers improved access to that data, Thibodeau said.