IE 11 Not Supported

For optimal browsing, we recommend Chrome, Firefox or Safari browsers.

Tennessee Emergency Operations Center Reports Lost Phone Recordings During Gatlinburg Wildfire

Part of the problem was an earlier failure of the system's automatic backup function.

(TNS) -- The state says it has lost the recordings of the telephone calls to and from its Emergency Operations Center on the day of the devastating Gatlinburg wildfire.

The following days' calls, however, were recorded and preserved.

The problem was discovered when state employees tried to retrieve information to fulfill open records requests by media organizations, said Dean Flener, spokesman for the Tennessee Emergency Management Agency, which operates the center.

The recordings might have provided critical information on how the crisis was handled. Flener said about 80 people equipped with 86 recorded phone lines were in the Emergency Operations Center in Nashville starting at 8 a.m. Central time on Nov. 28.

Those employees were responsible for coordinating fire departments statewide as they responded to Gatlinburg Fire Chief's Greg Miller's request for help in battling the flames that forced the evacuation of 14,000 people.

Representatives of more than a dozen state agencies were in the EOC arranging emergency shelter for evacuees and deploying more than 100 members of the state National Guard, the Tennessee Highway Patrol and state forestry firefighting resources.

Officials blame the flood of calls that day for the loss of the recordings.

Fourteen people died in the blaze and more than 2,500 structures were damaged or destroyed.

For seven months, TEMA officials labored to determine what caused the recording failure.

Part of the problem was an earlier failure of the system's automatic backup function. Flener said the backup stopped working "sometime in October 2016" but emergency personnel in the state EOC had no idea that had happened. Someone would have had to manually search the digital archive file to realize that calls were not being stored.

The system installed in 2012 didn't issue alerts when something was amiss, he said.

"Without the call recordings, there is no way to estimate the number of sent and received calls into the SEOC on Nov. 28, 2016," Flener said.

TEMA is not required to record calls to and from the state EOC but does so for training purposes and information verification, Flener said. The EOCs in Gatlinburg, Sevier County and Knox County do not record telephone calls.

When TEMA communications staff tried to retrieve recordings in response to public records requests, they found "there were no recorded phone calls in the telephone recording system archive prior to Nov. 29, 2016," TEMA Director Patrick Sheehan said.

TEMA staff and employees from a telecommunications maintenance contractor "investigated the issue multiple times," the director said.

"Neither TEMA telecom staff nor the vendor could determine whether the missing calls were actually somewhere else in the system or if the calls were not recorded," Sheehan said.

Sheehan hired a private company experienced in digital forensic analysis to recover calls and determine what caused the recording failure.

Flener refused this week to identify the firm contracted through state procurement rules.

"The company doesn't want to be identified because the analysis isn't complete and the final report isn't in and we will honor their wishes," he said.

"We are awaiting submission of final costs from the forensic data firm," Flener said when asked about the expense.

Employees of the private firm concluded that "because of the overload, there was not room in the recording system to preserve calls before Nov. 29, 2016," Sheehan said.

"There is a finite amount of storage in the system," Flener said. "As the system runs out of storage, the oldest calls are deleted to make room for newer calls."

The forensic analysis could recover only three calls from Nov. 28.

The first of those was at 9:44 p.m. Central time with a news media outlet, Sheehan said.

The second was at 10:31 p.m. between the TEMA Watch Point and North Carolina’s Emergency Operations Center about reaching Sevier County. Watch Point, Flener said, is the position covered 24 hours a day to field emergency calls.

The third call was at 10:32 p.m. with the Tennessee Department of Health EOC representative regarding nursing homes in Sevier County.

"It does appear that all calls prior to these three are missing from the system," Sheehan said.

Discussions between the state EOC and the Gatlinburg EOC probably didn't occur on the recorded lines in Nashville, Flener said.

While the EOC in Nashville coordinated resources requested by Gatlinburg and Sevier County, the authorities in the Gatlinburg EOC were responsible for local deployment of emergency personnel. Fire Chief Miller was the commander in the Gatlinburg EOC and oversaw all firefighting resources in the city and county.

Flener said most of the calls with Gatlinburg were "mobile phone to mobile phone." They dealt with the status of the wildfires, the need for resources and the use of the Integrated Public Alert and Warning System, IPAWS.

Miller had issued requests for help as early as noon because of concerns that high winds would drive the wildfire burning in the Great Smoky Mountains National Park into the city. Those winds toppled trees into power poles about 6 p.m., igniting fires across the city.

At about the same time, windswept flames from the wildfire in the National Park joined the destruction. The tally was more than 2,400 homes or businesses in the city and Sevier County damaged or consumed by flames.

When Miller at about 8 p.m. wanted to use IPAWS to issue a mandatory evacuation of the city, he had to have TEMA officials activate the alert. IPAWS was adopted by the Federal Emergency Management Agency as a way to reach all mobile phones within a certain area with an alert.

When TEMA officials tried to reach Miller about 8:30 p.m. to confirm the message the chief wanted sent, cellular service in Gatlinburg had been disrupted. Unable to reach Miller, TEMA did not issue the evacuation order.

Sheehan is awaiting a final report from the forensic data company that could recommend how to prevent a recurrence of disappearing emergency calls.

"The forensics firm has noted to TEMA its conclusion that the calls were lost due to the file buffer being full, and all the storage space available was only able to hold the calls going back to the end of the day on Nov. 28, 2016,” the director said.

Flener would not disclose how many days of calls prior to Nov. 28 also were lost.

He said TEMA's telecommunications staff periodically cleared calls from the system to make room for new recordings. This process was performed "once or twice a year."

"Prior to Nov. 28, 2016, the primary folder was last cleared around Nov. 18, 2016," he said.

Sheehan on Nov. 10, 2016, had ordered the state EOC activated when he declared a state of emergency because of the severe drought and wildfires raging across middle and east Tennessee. Flener said the recording equipment was constantly on, so any calls coming into the center should have been recorded.

Tennessee has had a state EOC since 1978, Flener said. He was unable to say if previous recording systems had malfunctioned or been overwhelmed by calls, but said the current system had not faltered before the Gatlinburg wildfires.

"Because of how TEMA uses its recording system, for training and information verification, there has never been cause to check or backup the recording system during an emergency activation," Flener said.

In addition, he said, "this is the first emergency TEMA has had where phone recordings have been included in open record requests."

Flener said TEMA anticipates the analysis of the system conducted by the forensic company "will help us to evaluate a proper course of action" to correct whatever failed in the current recording system.

Gov. Bill Haslam, who committed to a full and public review of TEMA's performance during the Gatlinburg wildfires, is aware of the state EOC recording problem, according to his spokeswoman, Jennifer Donnals.

Haslam "supports TEMA’s efforts to recover those recordings and the work of the data forensics firm currently trying to determine what happened," Donnals said.

Sheehan in May reached agreement with FEMA for the the federal agency to compile an after-action report on how the state agency handled all the wildfires that erupted in Tennessee during the drought.

The director wanted an overarching review of how all the various wildfires were handled and not just the Gatlinburg disaster. The Gatlinburg wildfire was the only one with fatalities.

That review, Flener said, should not be hampered by the lack of recorded calls in the state EOC.

"We have never used recorded telephone calls in prior after-action reviews," he said. "So, the after-action review for this event won't be affected."

Flener said FEMA has not indicated how long the review will take. There is no cost to the state for the federal review.

©2017 the Knoxville News-Sentinel (Knoxville, Tenn.) Distributed by Tribune Content Agency, LLC.