One of the Virginia's mainframes experienced a hardware failure around midnight Tuesday night and staff began diagnosing and fixing the problem. By Wednesday officials realized there were two hardware failures.
"We have determined the problem was with a controller that connects the mainframe to other systems, not with the mainframe itself," said state CIO Lem Stewart in a statement yesterday. "We will continue to monitor systems very closely and will install redundant hardware during scheduled maintenance on Sunday."
The backlog in data processing impacted state agencies that use the mainframe, including several of the larger agencies -- DMV, Social Services, Tax, VDOT, Employment Commission and others. Some agencies are having more challenges than others.
"We apologize to state agencies and citizens for this disruption of service ... and ask that citizens be patient with state employees who are unable to perform their work because of this disruption," the state said in a press release.
The partnership between Virginia Information Technologies Agency (VITA) and Northrop Grumman to transform the state's IT infrastructure should prevent such problems in the future and reduce the amount of time required to correct problems because of the installation of redundant systems.
"We have known for several years that the state IT infrastructure needs work" which resulted in the partnership with Northrop Grumman, Stewart explained. "VITA and Northrop Grumman are working to upgrade the state's IT infrastructure, including mainframes, servers, PCs, messaging and e-mail, and to construct a new data center in Chesterfield County and a back-up data center in Lebanon in Russell County. We are making significant progress and will continue that work."