Florida Department of State CIO Larry Aultman implements a drastic, six-month shift to the cloud for the state's beleaguered elections system.
They say that death and taxes are the only certainties in life. But Florida Department of State CIO Larry Aultman might add elections to that list. “Believe it or not, there is an election somewhere in Florida pretty much every Tuesday,” he quipped before an audience of peers at last month’s U.S. Public Sector CIO Summit, hosted by Microsoft.
Aultman took over IT at the Department of State in July of 2011, in the midst of negative media attention resulting from a hack on the state’s election system. While that particular issue centered on data from only a couple of Florida’s 67 counties, the media attention on the state’s elections process was unwelcome by Florida officials whose memory of the 2000 presidential election was still fresh.
Florida’s paper-based voting system led to weeks of uncertainty as voting officials struggled to interpret voter intent from ballots with hanging chads. Media satellite trucks lined streets in Tallahassee as the nation and the world waited to hear whether George W. Bush or Al Gore would be awarded Florida’s 25 electoral votes and therefore win the U.S. Presidency.
The controversy, which cast aspersions on the legitimacy of the entire election, led to the passage in 2002 of the Help America Vote Act (HAVA), requiring all states to begin using computerized voter registration systems, and enact voting system upgrades, by 2006. Florida received federal funding to improve their systems in 2004, but project challenges forced a hasty do-over that met the letter of the federal law but did little else.
According to Aultman, the new Florida Voter Registration System (FVRS) did its job in 2006 and 2008, but relied heavily on personnel to maintain functionality. The state legislature required many changes after the 2008 election, and the U.S. Census in 2010 brought about redistricting throughout the state, which also had major impacts on the administration of elections. Sweeping changes again came from lawmakers in 2010 in the form of state law 1355, which changed the way the state interacted with counties.
“We had to start tracking all of the third-party voter organizations and all of the activities in the counties for the voter rolls,” Aultman said. “It was a lot more information that we had to start reporting to the legislature and the Governor’s office and the court system that we had not done in the past.”
These changes required that Aultman and his team write a lot of new code before the 2012 election, which would take place on January 31 – barely seven months into Aultman’s tenure with the Department of State.
Aultman inherited a highly fragmented IT environment – the people and equipment funded by HAVA dollars, and everything else. “Probably the newest piece of software and system that we had was acquired in 2004/2006, but it was acquired to do exactly one thing, and that was satisfy the HAVA law,” he explained. “The law was significantly changed, the requirements and the reporting were significantly different, about all that we could really count on it to do was to keep up with who can vote and who cannot, which by the way, was its intended purpose.”
In addition, Aultman inventoried all the legacy databases and applications, many of which were written by staff who no longer worked for the department. He encountered very little automation in a pieced together legacy system that was at its end of life -- with ineffective back-up and virtually no capacity for disaster recovery. With lean staffing, a limited budget and an additional charge to consolidate the technical workforce whose skills didn’t necessarily align with the future needs of the department, Aultman concluded that a radical solution was in order.
Having recently completed a project for the Florida House of Representatives using Microsoft’s Azure cloud computing infrastructure, Aultman decided to take the state’s election system to the cloud. Azure offered three distinct advantages that made it an ideal fit for the project.
Practically speaking, the cloud option was much faster to implement than an on-premise solution. Making the decision to go with Azure in mid-August, the account was established in October 2011. Aultman and his team used the time in between to start working on coding the many legislatively mandated updates that needed to be in place before the January election.
Secondly, Aultman had experience with other successful cloud implementations, some using Azure. Lastly, the solution met the Department of State’s requirements for compliance out of the box. Aultman explained that most voter registration data is public information, so his security requirements may not be as stringent as other agencies.
With the support of the Secretary of State, Aultman invested in training for his existing workforce to ensure they’d be prepared to code the needed changes. He also made a successful case for hiring additional highly-skilled “top guns,” with salaries to match, who added significant value to the process.
On election night, tensions were high and all eyes were on the new system’s public reporting site, which had been load-tested to handle as many as 13 million page views within a 4-hour window. County election officials were poised with fingers on the send button as the polls closed in their jurisdictions. Seven seconds after the deadline, with 15,000 election followers hitting the refresh button per second, election results data had yet to materialize on the Department of State’s website.
Aultman describes a tense phone call from a state official, asking about the data, prompting him to grant assurances using his best “airline pilot’s voice.”
Three seconds later, the shouts of Aultman’s staff in the background confirmed that their efforts had paid off. “It was there and we did it, and for the rest of the evening, there was no problem at all,” he said. “It was a heart stopper, but it was awesome.”
According to Aultman, management is often in a vacuum where IT is concerned, but he’s a firm believer in communicating early and often, without sugar-coating the truth. He implemented structured monthly reporting to keep officials abreast of current priorities and projects.
“The benefit [of good communication] that most IT people I think miss is that once you have that close relationship, they want a win too -- they will help you, and they have means that you don't know about and are talking to people that you don't know,” Aultman said.
Internal communication is just as important. Consolidating staff from different divisions meant establishing a clear vision and communicating truthfully and regularly about expectations. “The culture had to be reinvented to be an open, sharing mentality.”
Photo: Larry Aultman, CIO, Florida Department of State