In the futuristic sci-fi movie Minority Report, actor Tom Cruise plays John Anderton, a detective in the police "pre-crime division." Anderton's detective work is a little different than today's -- wearing special gloves with sensors that cover his thumb and index fingers, and using hand gestures and voice commands, Anderton knits together disparate pieces of information from several databases by plucking the data from several plasma screens and dropping it into a central screen. After collecting all the pieces of data, Anderton rearranges them with his hands into a coherent file.
The scene is visually exciting and appears to be another special effect dreamt up by the wizards of Hollywood.
Or is it?
Not So Far-Fetched
At Pennsylvania State University, researchers are working on something similar, combining GIS, natural language technology, cognitive engineering and the relatively new field of gestural science to create their version of the computer used by Cruise in Minority Report.
The difference is the technology will help governments better manage crises such as hurricanes, terrorist attacks, disease outbreaks and forest fires.
The high-tech wizardry behind the project is fascinating, but its purpose is basic: to help people work together better while making decisions as a crisis unfolds. Michael McNeese, one of the project investigators and a professor at Penn State's School of Information Sciences and Technology, defines a crisis as a series of ill-defined situations with information flowing in from many different sources.
"The problem in crisis management right now is information overload. It creates cognitive problems for the person trying to interpret the data and make decisions," he said.
The other problem facing crisis teams is their lack of skills in manipulating and analyzing spatial data using a geographic information system.
"Specialists in other domains can't access GIS when they need it the most," said Penn State geography professor Alan MacEachren, director of Penn State's GeoVISTA Center. "They have to rely on GIS experts, which isn't very efficient."
GIS is a critical element to managing data during a crisis, yet few local and state governments have the funds to cross-train crisis team leaders in intricate geospatial mapping techniques, according to MacEachren.
A significant problem is that current GIS technologies aren't designed for the end-user or for use in team efforts, such as crisis management, MacEachren said.
"There's been no study in a systematic way to predict what works and what doesn't in these situations," he added.
The center's mission is to coordinate integrated GIS research with emphasis on geovisualization.
MacEachren and McNeese are part of the GeoCollaborative Crisis Management (GCCM) project, a Penn State University research initiative that the National Science Foundation funded with $400,000 as part of its Digital Government Research Program.
MacEachren said researchers are currently using ArcIMS from ESRI.
"However, our interfaces approach and software is not dependant on a particular GIS," he added. "We are currently in the process of implementing a version using open source GIS software, relying on Geoserver + GeoTools."
McNeese, who spent 23 years in the Air Force designing command and control centers, is an expert in the arcane field of cognitive engineering. MacEachren is a professor of geography at Penn State and an expert in geographic visualization and cartography.
Rounding out the interdisciplinary GCCM team are professors Guoray Cai, an expert in human-geographic collaboration, and Rajeev Sharma, a specialist in the science of gestures for human-computer interaction. Researcher Sven Fuhrmann, who works in the field of geovisualization and cognitive science, is also a member.
Together, the team is spending three years studying how teams of government workers respond to crises collaboratively, and developing technology that will enable them to synthesize geographic and other forms of data without having to become GIS experts.
Helping them are several government partners who are experts in dealing with natural disasters, hazardous spills and other environmental problems, as well as agencies that deal with terrorism.
Federal agencies include the Environmental Protection Agency, the Department of Homeland Security and the Federal Emergency Management Agency. State partners include Pennsylvania's Department of Environmental Protection and the Florida Division of Emergency Management. GCCM's business partner is Advanced Interfaces, a small software firm specializing in multimodal interfaces for GIS.
So far, the team has spent most of its time studying how crisis teams work in the real world and developing a way to permit human-computer interaction with geospatial information to occur in a group environment. The goal is to come up with a way that allows what the researchers call a new computing paradigm based on "multimodal," dialog-enabled interaction with geographic information.
The solution is called Dialogue Assisted Visual Environment for GeoInformation (DAVE_G).
"DAVE_G can recognize gestures in conjunction with dialog and interpret the meaning," McNeese explained.
Combining gestures and dialog is important, according to MacEachren.
"Spatial concepts are vague for computers," he said. "When a user tells a computer a location is 'near,' 'between' or 'north of' something, it has trouble interpreting what that means. But when you add hand gestures, the accuracy improves."
The technology is meant to work with large screen displays. For example, a crisis team tracking a hurricane might say, "Let's look at the population distribution here in the southeast" and gesture at a map on display, circling the region of interest.
The computer can interpret the combination of voice and gesture commands, zoom in the area and act on the next series of queries. The team member might gesture to indicate the possible track of the hurricane and ask the computer to display what areas would be most affected by flooding if the storm tracks north or south of the current location.
On a smaller scale, GCCM is also developing software to let field workers manipulate GIS software on tablet PCs and PDAs using styluses.
More Work To Be Done
In terms of practical use, GCCM is testing the capabilities of DAVE_G for hurricane disasters and West Nile outbreaks. The team is also working on a demonstration project for the Port Authority of New York and New Jersey that will aid the agency with possible oil and hazardous material spills.
With the research project just halfway completed, it's too early to judge the results. But if things go according to plan, GCCM and its multimodal software for human-computer interaction could change the way groups of government teams work together and make split-second decisions during a crisis.
"This technology will make the information that's essential for crisis management more accessible to the people who need it," said MacEachren. "Local police, fire and emergency management personnel are the ones who need to know where things are and make decisions based on that information. But few, if any, of these people are highly skilled in analyzing geographic information with software. This technology will make it possible for these people to use the information without expensive training."