"We've had a problem in the state, which I'm sure is generally found around the country, of actually losing some of our electronic records," Reed said. "We have e-mails from a previous governor's administration that have disappeared. We also have Wang disks from another governor's administration that we don't have the right equipment to read what is on those disks. We currently have a situation where there's a considerable amount of public policy discussion, and, frankly, development, going on via e-mail, and we need to have a set up to be able to capture that for history."
Reed said it's his office's responsibility to keep the records, because they are historical documents, from both the state and the state's local governments.
The irony of losing electronic records at a time when storage is getting cheaper is not lost on the Secretary of State's office, said Steve Excell, assistant secretary of state, and the project leader of the Digital Archives.
"If we had tried to do this five or six years ago, we would've been doing a lot of customized programming and using a lot of customized equipment," he said. "Now, it can be done pretty much with off-the-shelf equipment. If the unit cost of storage keeps dropping, we'll have some interesting choices ahead."
Excell said the $14 million archive building will contain a scalable SAN that, right now, is capable of storing a pedabyte of records -- or one quadrillion bytes -- and should be able to hit between 10 and 800 terabytes in the future.
"Our customers are state agencies, all three branches of government as well as local government agencies," he said. "We're looking at something that will allow us to grow as fast as our customers demand."
The plan is to start with robotic tape storage that will allow the office to store the data offline. But, as the cost of online storage goes down, the state may consider moving to online storage of the electronic data.
The Digital Archives building is slated to go online in the first quarter of 2004, Excell said.
"In the meantime, we'll be staffing up and working on the procurement of the computer systems and seamless desktop delivery systems so we don't have to turn every state and local government employee into an archivist," he said. "We're looking at ways of auto archiving, applying data-mining tools to figure out what to keep, eliminate duplicates and then to create a platform-neutral storage environment so we can retrieve things not only decades from now, but archivists think 100 years from now."
Excell said the office will be using is related to meta data, XML and various types of wrapper technology to create an authenticated record that can be pulled back out of storage at any time.
"We are dealing with the very history of our state and of our local governments in the state," Reed said.
