Washington State GIS Transportation Repository Takes Long Road

Six years in the making, the state and counties teamed to create the Washington Statewide Transportation Framework -- a massive project that required data integration and conversion, software integration and application development.

by / December 16, 2009

A small GIS team at the Washington State Department of Transportation (WSDOT) seems to like a good challenge. After all, it's not every day a state DOT tries to create a single, seamless repository of transportation information covering the entire state -- from the small private road to the primary arterials -- and does it successfully.

But that's exactly what WSDOT is doing, in collaboration with Washington state, in building the Washington Statewide Transportation Framework (WA-Trans). It's the data-heavy system Tami Griffin, WA-Trans project manager, and her advisory team envisioned it would be six years ago when WSDOT decided to create a comprehensive, statewide transportation GIS. As she suspected, the development and implementation of WA-Trans has become one of the most complex and progressive GIS transportation initiatives ever undertaken. The project is a massive Web-enabled GIS transportation system -- a huge product of data integration and conversion, software integration and application development.

"Across the states there are some DOTs that are implementing some of the smaller-scale pieces of what WA-Trans has, but I am not aware of any agency that is implementing a system on the same scale as WA-Trans -- and only a few are thinking of something even close to this," said Griffin, WSDOT's IT and GIS project manager.

With the primary goal of creating a single-lane "highway" to one transportation data repository for the entire state, WA-Trans incorporates the information management strength of GIS software with the real-time data transformation and distribution capabilities of Web-based GIS and spatial extract, transform, load (ETL) tools. The combination eventually will provide users with a standardized, seamless and holistic view of Washington's transportation network. Initially connecting the state's and counties' road-related data, WA-Trans' final destination will be multimodal -- including continuous and connected data sets for light rail, heavy rail, ferries, ports, airports and nonmotorized transportation infrastructure.

By defying the odds -- there was no precedent for such complexity -- WA-Trans is moving toward better efficiency, collaboration and fiscal responsibility for local and state authorities. It's also a testament to what a small group with limited resources can achieve when committed to a common goal.

"Our transportation network is one of the backbones of our society," said Griffin. "Fragmented backbones don't operate very efficiently. We wanted WA-Trans to connect our data divides to foster better collaboration and improve operations and services, and ultimately our bottom lines. Though we've really only begun with WA-Trans, we've already conservatively estimated that the [return on investment] to the state will reach $26 million over 20 years."

A Transportation U-Turn

WSDOT is charged with monitoring more than 18,300 miles of state highways and 3,600 bridges -- including the four longest floating bridges in the U.S. In addition to building, maintaining and operating the state highway system, WSDOT is responsible for the state ferry system -- the largest vehicle-ferry system in the world - by ensuring its 23 vessels safely and efficiently carry 24 million passengers annually. The agency is also in its fifth year of a 20-year capital construction program that will deliver more than $15 billion in projects, including $11 billion for 390 highway projects.

In parallel with the robust growth of its state roadway network, WSDOT geospatial personnel have become avid users of advanced GIS technology. It helps them collect, create, inventory and maintain their detailed transport data stores, and leverages that information to enhance other business analyses and processes.

Many of the state's 39 counties have mirrored WSDOT's spatial data management initiatives to help them acquire and maintain the most up-to-date and comprehensive view of their countywide transportation information. Many counties have been mapped extensively, and county authorities have benefitted greatly from the rich data sets.

However, this parallel approach among the state and local counties also has been problematic. Although each has

amassed valuable, critical transportation databases, they have done so with their own singular view and singular purpose: WSDOT focuses on state roads, while "County A" focuses on its county --- ending its data coverage at its county border. That has led to a winding and bumpy road for data sharing and intergovernmental cooperation, particularly in the emergency response arena -- where incidents often cross jurisdictional lines.

"Trying to fill in data gaps during an emergency is difficult," said Michael Leierer, assistant project manager and WA-Trans technical lead. "But even in nonemergency situations, local governments needing to do regional work have had to manually request data sets from other counties, and when the data arrived it was not in a format they could read. That makes it difficult to collaborate."

Washington's GIS community recognized that this insular approach was a significant roadblock to operational and financial efficiencies at the state and county levels. What was needed instead was a centralized, statewide transportation data set that mimicked the real-world, continuous and connected roads, railways and ports that authorities view from their office windows.

All Roads Lead to 'How'

But was such a system feasible? Putting forth such an objective would spark bedeviling "how" questions: How do we elicit the participation of the local counties? How do counties easily, securely and routinely provide spatial data? How do users quickly extract the data layers they need? How do we standardize diverse attribute data, such as addressing and linear reference systems? How do we ensure that the centralized database is maintainable and scalable for future growth? Very few, if any, state DOTs had asked these questions before, let alone answered them. In fact, WSDOT considered these questions before but hadn't been able to adequately resolve them.

In 2003, the agency assigned Griffin to specifically revisit this vision. Her approach was to first address the critical "why" and "who" questions.

"It was clear to me that if Washington state was going to develop a centralized, statewide transportation framework, we first needed to decide why we're doing it and who we want it to serve," Griffin recalled. "Once you know that, then you can start to figure out the 'how.'"

Griffin began by forming a steering committee with representatives from 13 state and local authorities -- including some geospatial professionals -- as well as a partners group. She then initiated an extensive business-needs assessment. Interviews were conducted with various organizations across the state to assess their interest in WA-Trans; they were asked to identify and to rate the most important business functions they wanted the system to address. Based on that feedback, the WA-Trans team, steering committee and partner group developed a prioritized list of business needs (e.g. topping the list were homeland security, emergency management, planning, traffic safety and maintenance). They became the core guide for driving the framework's development.

"Assessing and prioritizing the business needs was a great way to bring all the different parties together to agree to and commit to the project," said Griffin. "Because everyone had a chance to collaborate -- to voice their needs and priorities -- they completely bought in to the assessment and the outcome. Everyone felt represented, which was crucial."

Having a clear guide also enabled them to search for technical solutions that would meet those needs. Griffin looked to other DOTs such as Oregon and other transportation authorities to see if there were existing data models that could work for them. After comparing a few options, the project team chose to base its WA-Trans data model on the Oregon structure.

With the base data structure, critical data layers, functionality and application tools for WA-Trans identified, the four-person project team could begin the arduous task of designing, building and testing the system -- a journey that put the

WA-Trans crew in overdrive for the past four years.

Building the WA-Trans Frame

From the beginning, the team wanted to adopt the transportation framework in a literal sense, meaning the state would construct the standard structure of the WA-Trans "house" and the counties would be the individual interior designers. They chose this design for two reasons: First, local authorities typically maintain the best local data sets, so incorporating the county data would ensure that WA-Trans offers the best available data. Second, local governments would resist participating in WA-Trans if they first needed to change or convert their data.

However, providing the framework house meant that the system had to integrate and normalize the disparate county data into a centralized database - and it had to allow authorities to work in their native GIS software and formats. In addition, they needed to have a back-end system that could effectively manage, integrate and manipulate a multitude of data layers as well as a user-friendly front-end Web portal for submitting and extracting data. Griffin and Leierer said the only way to resolve this interoperability and data-delivery challenge was with a spatial ETL tool that could transform and deliver data in user-specified formats.

"It was essential that we developed processes to ensure data providers' GIS and tabular data were placed into the WA-Trans database accurately and that they preserved the provider's original meaning," says Leierer. "We searched for translation tools as well as ETL tools, but they typically only work on tabular data. We needed a complete spatial ETL solution that offers data integration, translation and transformation of disparate data sets, as well as an electronic delivery mechanism."

The WA-Trans team chose Vancouver, British Columbia-based Safe Software's FME, a spatial ETL solution that enables GIS professionals to translate, transform, integrate and distribute spatial data from more than 225 formats.

To determine whether the conceptual architecture would not only resolve the data-interoperability equation, but also would provide the means to continually maintain the data, Griffin and her team launched a pilot project in 2004 with Pierce County, King County and the Puget Sound Regional Council to evaluate the initial structure of WA-Trans.

For eighteen months, the pilot counties provided tabular and spatial-based roadway data for the WA-Trans team to integrate into the system. Of critical importance was that the data be transformed accurately into the WA-Trans standardized data model without adversely affecting their original data. This capability would ensure local authorities could use the WA-Trans data as a base spatial resource to perform critical business functions, such as address geocoding and transportation planning.

"We first gave each county access to their own data sets in WA-Trans to let them analyze the accuracy and completeness of their data," said Griffin. "Then we integrated both counties' data layers using the counties' predetermined 'agreement points' (previously agreed positions of overlapping map features) to see if they could extract and integrate data layers from each other. It worked very well and the feedback really helped us fine-tune the translation processes."

It was an important proof-of-concept result for the WA-Trans team and data providers, and validated WSDOT and the numerous counties and states that had been intently monitoring and awaiting the initial verdict on WA-Trans. It was also an important technical exercise to confirm FME's ability to handle the system's data-transformation, maintenance and delivery demands.

With the core infrastructure in place, Griffin and colleagues began inviting other counties to participate in WA-Trans. Initially soliciting four other counties (including Kitsap and Walla Walla), WA-Trans presently contains road-related data from eight counties and plans to include 16 of the state's 39 counties by spring 2010. The remaining counties will require more time and funding.

Navigating the Data

To prepare for the influx of new data, Leierer led an effort to create data processing

routines that would be as automated as possible. But the routines had to be individualized for each provider's unique geographic coverage, scale, datum, metadata, linear referencing, attributes and data quality. Using FME, the WA-Trans team developed a "master" transformation template to manage processes common to all providers, to place provider data into the centralized database, and to flag processes needing customization for each provider.

To submit data to WA-Trans, providers securely log on to the provider portal and browse their own system to select data files for upload. The portal validation system verifies that all required GIS files are present; then the provider is allowed to submit the data. Submitting the files initiates FME to begin the validation and translation process that's specific to that individual provider. After the data is successfully translated into WA-Trans, automated change detection processes can flag changes since the most recent data submittal. FME also performs control checks to verify the proper data schema, the completeness of data; it also identifies data duplicates and flags the possible data issues. Once the data passes the quality assurance process, it's distributed to the WA-Trans Microsoft SQL, ArcSDE centralized database that all providers can access.

"By building the FME master transformation template and other customized templates, we have been able to integrate local data sets much more efficiently, reducing the hours per individual provider by 90 percent," says Leierer. "And we continually ask for provider feedback each time a new data set is added to WA-Trans."

To retrieve WA-Trans data, users simply access the data-user portal built on Latitude Geographics' Geocortex Essentials technology, search for the specific data they need and choose the data sets in their desired format. FME then extracts the chosen data layers, transforms them into the specified format, and when the data is ready it automatically sends users a message with a link to their downloadable zipped file.

"FME is critical for bringing data from providers into the database, and for serving it out," said Leierer. " Having a good spatial ETL tool like FME is a critical factor in WA-Trans's success thus far."

24-Hour Drive-Through

With WA-Trans a reality, WSDOT personnel, other state agencies and county authorities are navigating a path to numerous operational and financial benefits, and paving strong collaborative bonds along the way.

"WA-Trans has facilitated dialog and collaboration between counties and entire regions on a scale rarely achieved with previous data-management structures," said Griffin. "With WA-Trans, suddenly everyone can be on the same data page, enabling people to perform E-911 planning and routing, transportation planning and maintenance, root cause incident analyses and road-related reporting with far more efficiency and confidence. And they can do it on a countywide, regionwide or statewide basis."

By spring 2010, WA-Trans will include seven of the most populous counties, which Leierer said comprise more than 70 percent of the state's traffic incidents. That data will enable users to analyze the root cause of traffic incidents, which will be particularly beneficial to WSDOT and local jurisdictions. For example, in the past, individual counties as well as WSDOT were only able to analyze roadway incidents within their respective jurisdictions, which didn't provide an accurate view of the entire safety mitigation picture. WA-Trans provides a holistic base map. Now local law enforcement can identify the exact X,Y location of collisions on the roadway, and WSDOT personnel can combine those geolocations with other roadway-related information, such as the locations of bars or schools, to determine if there are other mitigating factors contributing to the volume of collisions at a given location. That can help the state spend their transportation dollars more effectively, rather than using anecdotal information for justifying decisions.

Access to the seamless database opens up other interests and potential applications. Regional planning and transportation planning organizations now can acquire county data directly

from WA-Trans rather than dedicating time and resources to create and maintain data themselves. The state's health department is interested in obtaining detailed bridge and culvert data to improve water-related management strategies. The departments of Social Health Services and Corrections are thinking about creating a system similar to WA-Trans that would help them understand spatial relationships to schools or day care facilities when they are placing sex offenders who are discharged from prisons.

The new transportation framework will greatly ease WSDOT's compliance with new reporting requirements for the Highway Performance Monitoring System (HPMS) program -- an important funding and budgeting mechanism to maintain highways. In the past, DOTs only had to submit required state roadway information in tabular form. As of 2010, HPMS reports need to include spatial data down to the county level -- a requirement that almost all state DOTs are presently wrestling with, said Leierer.

As authorities continue add to WA-Trans, more benefits and applications will surely dot the landscape. And that means the WA-Trans team likely will remain in overdrive for a good while longer. But then again, they seem to enjoy the challenge.


Mary Jo Wagner Special to Government Technology