So before each hurricane approached, the IT staff dismantled the data center, boxed up the equipment and relocated it into a decommissioned water tank. With 8-inch thick concrete walls designed to hold 770,000 gallons, the water tank protected the technology. Because the equipment was disconnected; however, city staff couldn’t access their network.
“During the actual events of the hurricanes, there wasn’t Internet access, access to data or anything along those lines,” said Larry Di Gioia, director of information services for Altamonte Springs. “We’d come in the next morning and reassemble everything.”
The system disruptions and network downtime were a big disadvantage for citizens. “We had periods of time where we were literally down for multiple days and not able to service our customers,” said Anthony Apfelbeck, the city’s fire marshal and building official. The situation also precluded city departments from deploying new e-government services, he said. “We had no system reliability whatsoever.
Into the Dome
Following the 2004 hurricanes, the city enacted a multiyear plan for technology and disaster recovery planning that would provide continuity of operations during future storms and power outages. In June 2005, the city moved IT operations into the one place officials could rely on to keep it safe: the water tank.Known as the “Dome,” the water tank was the obvious place to relocate the data center, Di Gioia said. “It held water in for a number of years as it served as water plant No. 1 for the city,” he said, “I’m sure it can keep water out just as well.”
Before the data center moved into the Dome, the Information Services and Public Works departments worked with an architect to modify the structure. Buildings were added onto two sides of the tank, and the original water tank infrastructure was retrofitted. It’s now a fully operational office structure, with the additions acting as administrative offices and the water tank structure housing the server room/data center area as well as some IT staff members.
On June 24, 2005, after the water tank was retrofitted and construction was complete, Information Services staff moved the IT equipment across the parking lot and into the Dome — for the last time. “It took us about three hours to cut everything over. … And we were up and running,” Di Gioia said.
Thankfully the Dome hasn’t been tested yet by a hurricane’s force, and the city’s network has yet to experience any unplanned downtime since the data center relocation, according to Apfelbeck.
IT Awakening
Creation of the multiyear plan marked the beginning of an IT shift in Altamonte Springs. Di Gioia began working for the city in May 2003, and that year he identified numerous deficiencies within the Information Services division. Among other things, the division lacked a disaster recovery plan, a defined mission, adequate infrastructure and alignment with the organization.The city addressed one of those deficiencies in spring 2008 by opening the Public Works West Altamonte Facility, a hurricane resistant building that houses a backup data center. “Both [city data center] facilities are on dedicated generator tanks underground for fuel, full battery backup, surge protection, everything,” Di Gioia said. “All the generators are tested once a week, and we run off them. We also run our [disaster recovery] test twice a year.”
The disaster recovery site also is equipped with electronic door locks, video surveillance, fire suppression systems, as well as redundant heating, ventilating and air conditioning compressors.
Along with the new data centers, several IT upgrades have been implemented in the last eight years. Some projects and their benefits:
Virtualization reduced the number of city servers from 80 to 12 and cut maximum power consumption from 52,000 watts to 12,830 watts, both of which have resulted in cost savings.
A voice over Internet protocol telecommunications system was installed, providing one switch for voice and data.
The primary Police Department server package was outsourced to the county that Altamonte Springs resides in. “Rather than spending a million dollars on a new one, we partnered with the Seminole County Sheriff’s Office to house the data,” Di Gioia said. “We use their software, and it works fantastic for us.”
These projects have not only made the city more responsive to citizens’ needs, but they’ve also aided public safety. Police Cmdr. Robert Johansmeyer said a redundant fiber-optic loop ensures that all 16 of the city’s facilities never lose network operability. Di Gioia said the city leases fiber from Seminole County to be cost-effective. “We’re able to get great replication of the data, and that also gives us the ability to run continuous data protection,” he said.
In an emergency, police officers can now access their data and applications from computers in the city’s Emergency Operations Center, which is located in a different building than the police department, Johansmeyer said. “The end-user would never realize that connectivity was lost, or when we move locations, you would never notice that you’re working on a different computer,” he said.
Night and Day Difference
Changes to Altamonte Spring’s IT equipment and infrastructure have led to more convenient services for citizens. The biggest benefit has been reliability. “It’s confidence that we will serve the residents when they need us the most,” Di Gioia said.The network reliability allowed the Building/Fire Safety Division to deploy new software that moved some if its services online, such as filing building permits, inspection requests and payment of fees. “Those capabilities we couldn’t even envision seven, eight or nine years ago,” Apfelbeck said.
The division also is responsible for damage assessments following a storm. Before the IT upgrades, city staff had to write everything by hand when working in the field, he said. In the next year, the city plans to roll out GIS-based damage assessment software that Apfelbeck said wouldn’t have been possible without better network reliability and confidence in the system.
“Ten years ago, we would have been pulling our hair out trying to respond to customers after a two-day time frame of having no access to our building permitting software and them wanting answers on whether inspections have passed and whether or not we’ve issued a permit,” Apfelbeck said. “We wouldn’t be able to answer it. That was a frequent occurrence, and today it’s just an absolute night and day difference.”
In addition, first responders have seen their work process change for the better. Johansmeyer said that prior to the IT upgrades, officers couldn’t access the city network while on patrol. Now they use laptops to access the city e-mail system and the network, which houses policy, procedure and reference materials. And another project the city began rolling out in January provides the same access to police officers who ride motorcycles. “We’ve never been able to get them a reliable solution for that kind of technology,” Johansmeyer said. The motorcycle-riding officers will use netbooks and thumb-drive air cards to access the network, marking the first time they will have access to this data while in the field.
The overarching change to Altamonte Springs since the relocation of the data center, said those involved, is that IT deployments and projects have a defined mission and are successful because of the collaboration among departments. “It’s been a flourishing relationship all the way around,” Johansmeyer said, “and it just keeps getting better and better.”
