It took half an hour for Chevron to provide wind direction information to the county - information the county needed to decide which residents to warn.
Once the county had wind direction, a computer glitch caused another 30 minutes to elapse before the warning calls were sent out. The event is one of 51 major incidents in the last 15 years involving refineries or chemical plants in which the notification system failed in some way.
Since then, the county's contract with vendor Dialogic Communications ran out, and in April, it deployed a new system by Honeywell International.
This new system has performed well in limited duty on about four police calls, including a missing person's report, a gas leak and a grass fire. Notifications went out to more than 20,000 residents without a problem.
"We haven't activated the system through the refineries, but we've done it for several police incidents, and it's worked flawlessly," said Lt. Jeff Hebel of the Contra Costa County Office of the Sheriff.
Having a functional emergency telephone system that notifies residents during a potential crisis is a vital part of an effective emergency warning system.
Key Differences
Several key differences separate the new Honeywell system from the previous one. Chief among them, according to Art Botterell, Community Warning System manager for the Contra Costa Sheriff's Office: "It works."
The new system lets the county notify residents of a potential hazard by disseminating a computer-generated phone message to thousands of numbers accessed from the 911 database. The previous system needed a taped voice message, the creation of which wasted time. The new system translates a text message into a voice message.
Legally the notification begins when the county activates the system, but the refineries push the button first and notify the county. From there, the county fills out one of several templates to craft a message that will ring phone numbers in the affected area.
Templates include a general-purpose warning; one for hazardous materials incidents, which has more details about closing windows, filling gaps in doors with towels, turning off fans, etc.; one for evacuation; one to be on the lookout for a suspect on the loose; and an "all clear" message.
County officials bring up a map of the affected area on a computer screen and draw a polygon around the target area. The system queries the 911 database, calls up the phone numbers of all residences in the target area and sends the message.
It takes about 85 seconds for the new system to send a call and receive feedback on whether the call reached its destination. That's a big difference from the previous system, where feedback was delayed by hours, making it useless.
"At that point [when the calls are made], we've got multiple servers generating a lot of phone calls, and then - and this is where it gets dynamic - you have to try to determine how many calls the local telephone network can absorb on this day in this area," Botterell said. "And that number is highly variable."
That's where the feedback comes into play.
The feedback reports on which areas can absorb the phone calls, and which returned busy signals and need to be re-sent.
"Right now, we're at about 85 seconds, and we're trying to get that down below a minute," Botterell said, adding that the feedback starts coming back seconds after the notifications begin going out, which allows officials to resend calls that didn't go through.
"Mostly [the old system] lacked any ability to get the call detail back in real time," Botterell said. "We would get a report back in a couple of hours, but there was really nothing we could do with that information at that point. It wasn't a feedback loop; it was just sort of an after-action report."
Multiple Messages
The telephone system is really a subsystem within the community warning system that includes sirens, the Emergency Alert System (EAS), the weather service radio alerts, cable TV alerts and AM radio broadcasts. Any one by itself is not enough, and each is an important part of the larger warning system.
"If you look at the social science research that's been done on how people process warnings, you come to one of these conclusions that's utterly obvious and common sense once science proves it," Botterell said, "and that is that people hardly ever act on a single warning message. Everyone has experienced false alarms, and nobody wants to look a fool."
When people get the same warning messages multiple times from say, the radio, the TV and a telephone call, they begin to take the warnings seriously, Botterell said. The new telephone system is part of an overall new notification system that is a combination of pieces to be deployed simultaneously.
"It's all driven through our common alerting protocol - CAP - which is a beta standard that allows us to build kind of a warning Internet so the telephone notification system becomes one particular peripheral of the network," Botterell said. "The telephone system was the one piece of technology that hadn't been fully integrated into the larger warning system."
The main difference between the new system and the old one is that it's activated by a Web service interface between Honeywell's computers and the county's.
"Our computers talk to their computers on a very sophisticated level, and pass statistics and all sorts of information back and forth," Botterell said. "And that made it easy to integrate the system into the overall common alerting protocol-based control system we're building."
It's early, and the real tests will come when a refinery sounds a warning, but county officials are confident the system is rounding into shape.
"There is a level of integration and sophistication that we just haven't had," Botterell said. "And did I mention it works? Reliability is huge."
