No Child Left Behind (NCLB), a federal law to improve public education, has been criticized for only measuring student performance at a single point in time, rather than for the duration of a child's academic career. But that problem may be changing. For perhaps the first time in history, technology has the potential to not only enhance student learning, but also create a comprehensive framework for measuring and assessing student performance at the district, state and ultimately national level.Determine the value-added of specific schools and programs by following individual students' academic growth;
The Brookings Institution, one of NCLB's critics, applauded a 2005 effort by the U.S. Department of Education (ED) to start a pilot program that measures annual growth in student achievement -- what they learn from one year to the next. At the same time, however, the think tank emphasized that this holistic approach is only possible if states have the data resources for this type of measurement model.
As a result, the law's data requirements have spurred state departments of education to implement more sophisticated and complex information systems. Pressure from all government levels to generate public school data has, in turn, heightened expectations of technology offices to provide infrastructure for data management. While some states have responded well to the business and technology side of NCLB compliance, other states are farther behind on the learning curve.
New Era of Accountability
After President Bush signed NCLB into law on Jan. 8, 2002, the acronym became a loaded household name known for its mandates on students, teachers, school districts and state departments of education. The law requires that states make public the names of those schools that aren't making the grade. Districts and states must collect, categorize and analyze attendance data, test scores, teacher qualifications and other school information. Those schools that do not demonstrate adequate yearly progress (AYP) according to various NCLB measures cast a negative shadow on their stakeholders and are at risk of state takeover.
NCLB serves as a measure of school performance and method for improvement. It requires intensive and nuanced data collection and reporting. In the four years since the 670-page law passed, states have been working feverishly to adhere by 2014 to the myriad requirements outlined in the ED's Road Map for State Implementation, focusing particularly on student assessment, disaggregation of data and proficiency.
The pressure to collect and assess student achievement data is high. If states don't implement an annual measurement system, the federal government can remove some of the $66 billion in funds or grant money it spends on K-12 education annually. While this is just a fraction of the more than $450 billion states and local governments spend, it's enough to force states to overhaul data collection and analysis.
Education data, if properly organized, aggregated and presented, can provide federal, state and local entities with a wealth of information for research and evaluation. This scenario is fast becoming the goal for state departments of education. To optimize data management, educators need the help of technology and business experts.
"Data collection and reporting requirements mandated by NCLB, in the past, were not a required component of any IT department's strategy," said Michael Droe, chief technology officer of Hacienda La Puente Unified School District in California. "As a requirement for every district to meet AYP, all subgroups must meet AYP as well. This requires data collection and reporting instruments that would not otherwise be available except through the use of technology."
The volume of information produced from these assessments and the ability to align these results with state assessments and standards to make "just-in-time instructional decisions" would not be possible without technology and rich data management tools, such as data warehouses and longitudinal systems that match each student's achievement records over time from pre-kindergarten through state college or university, Droe said. The ED is encouraging a move toward longitudinal data systems (LDS), which, according to a 2005 Data Quality Campaign (DQC) survey of state data collection issues from the National Center for Educational Accountability (NCEA), do the following:
Identify consistently high-performing schools so that educators and the public can learn from best practices;
Evaluate the impact of teacher preparation and training programs on student achievement; and
Focus school systems on preparing a higher percentage of students to succeed in rigorous high-school courses, college and challenging jobs.
In addition, the DQC recommended that states incorporate these overarching concepts when designing an LDS: student privacy, effective data architecture and warehousing, interoperability, portability, and professional development for users.
"Currently three-fourths of the states have the basic elements needed for a longitudinal data system," said Nancy Smith, deputy director of the DQC. "Most other states are in the process of implementing those features. A few states have the majority of the elements. Education staff and policymakers are beginning to see the power of longitudinal data beyond federal and state reporting and accountability," she said. "Strong efforts are being made to build data systems that inform financial and policy decisions at the state level and in the schools, and most importantly in the classroom."
Other states are trying to increase and enhance automated reporting, create more robust vertical and horizontal interoperability, and implement data warehouses to feed today's sophisticated reporting requirements. But according to the DQC, all states still need to improve how they collect data.
Power of Suggestion
The ED oversees states' progress under the NCLB. CIO Bill Vajda plays a central role in data management within the federal department and indirectly among states, and said he intends to bring tangible and distinct business results under NCLB. "We have to do our job effectively for funding oversight to work," he said. "NCLB is broadening the constituent base and increasing the demand for data and access."
Although states are left to gather and present their own data, the ED is increasingly advising them on how to best design their data management systems to meet NCLB requirements. Ross Santy, deputy assistant secretary of data and information at the Office of Planning, Evaluation and Policy Development, helps school leaders make sense of how to house educational performance data in conjunction with NCLB. "There is no standard guidance on cataloging or presenting because all state systems are so different, in terms of standards and assessments," he explained. "We utilize the information they do show and ensure they are reporting through monitoring visits."
The ED awarded 14 grants to help states launch their data systems, and through the Institute of Education Sciences (IES), the federal government's principal agency conducting research on education, plans to issue 50 grants in all. The department requested a $55 million increase in IES funding for fiscal 2007 -- one of the largest single increases asked for in the presidential budget, Santy said. "We are hoping this will help more states."
Another federal program, the Education Data Exchange Network (EDEN), helps states transfer and share education program data, and is meant to streamline data collection and grant reporting. "We are finalizing a single portal," said Santy. "Right now all state coordinators know where to go. Everything is still primarily flagged and evolving into a reporting tool."
The federal government is using EDEN to provide some information standards to department of education CIOs who are tailoring their own policies, explained Santy, by demonstrating what they are responsible for reporting, and how those larger requirements affect state level data. "We want to get to the point where states can report education data in the most efficient way possible, and then discover other things we can do with that data."
How are states doing? "We get a fair amount of information from the vast majority of states," he said. "Some have done a phenomenal job of mapping all the information and reporting it for the central repository."
All EDEN data, which was initially voluntary for states, date from the 2003-2004 school year. The ED is finalizing regulatory guidance to set into motion a two-year transition for all states to report using EDEN, starting in 2007. If a state can show an inability to provide that data or at the level of quality expected by the ED, Santy's office will work with them over a two-year transition period to make sure they are ready to report.
Leading the Pack, Longitudinally
The key piece of technology in all this is the data warehouse. Florida leads among states with its comprehensive student databases. "We started a little ahead of the curve because Florida has historically been a data-rich state when it comes to education," said Ron Lauver, CIO of the Florida Department of Education.
Also, Florida's top-down policy has long encouraged data collection for school improvement. Gov. Jeb Bush's pre-NCLB education initiatives, such as A++, already emphasized many aims of the federal law, such as annual measures and learning gains.
Florida's "K20 Education Data Warehouse" stores all data on individual students from their pre-kindergarten year through 12th grade, and links it with those students' data from community colleges and state universities. Any data in this central repository can then be used to generate administrative and funding reports, queries and research extracts, and "provide links to what kids have done and are doing," said Jay Pfeiffer, assistant deputy commissioner in Florida's Division of Accountability, Research and Measurement.
In addition to its main data repositories containing standardized and state test scores, Florida generates adjunct data to support education. For example, another warehouse stores data from other state agencies that impact children, such as those that handle foster care, vocational rehabilitation and juvenile justice.
Florida has also built new technology tools to enhance education. A system called Choices links education data to students who later enter the labor market, and another, SunshineConnections.org, is used by students to map out their curricular goals. In partnership with Microsoft, SunshineConnections.org provides state and local data to teachers and administrators statewide. Florida is also designing student performance profiles that will correspond to the state's geographic regions.
Florida is not alone in moving in the direction of longitudinal systems. In 2006, the Utah Legislature passed its Education Information Technology Systems law, requiring coordination between public and higher-education IT systems, including the use of a unique student identifier. Missouri is moving in that direction as well.
Missouri's core data collection system is Web-based with the capacity for users to insert edits to the material used by each of the 524 school districts. Six times a year, districts submit data to a state-run relational database that generates payments and determines school district compliance with statutes and regulations.
Another success story can be found in Georgia, which is running a data warehouse that tracks grades and test results for each of the state's 1.3 million students, from year to year and from school to school.
Is It Working?
Despite some successes, many states struggle with the technology equation. The problem lies primarily in the area of collaboration, or lack thereof, according to Tom Ogle, director of School Core Data at the Missouri Department of Elementary and Secondary Education. "[NCLB compliance] is a collaborative venture between local school officials, districts and the state," he said. "If there is a link missing in the chain, the system won't work."
Even when collaboration is present, Ogle often hears the word "overwhelming" from employees. "There is a lot to do in a short period of time in terms of collecting and evaluating data. Humans can only do so much, and we're definitely putting stress on existing systems."
Santy has observed a leadership and communication challenge among states and within the federal department. "On the state level, it's knowing who -- i.e., the state IT offices -- needs to be reporting to the federal government on NCLB data management. We work with a fair mix of technical and data file builders, and some program people in the accountability offices. In our own office, we try to connect folks who work with the data to the programs. We have the same issues and busting down of silos here just as states are struggling with the same thing."
For some states, NCLB may have been a shot in the arm to coordinate data centers. On the other hand, Ogle, who has been working with the Missouri Department of Education for two decades, has not seen an altogether drastic change in data management due to NCLB -- with the exception of school-level data collection. "We have long had leadership in the state department and districts that emphasized the importance of having quality data collection," he said. "There's a long history, and we have been collecting individual records from teachers for 18 years. However, with the EDEN system and NCLB, the stars were aligned to change the culture of technology. We bought into moving past individual data collection to a centralized system early on, just as we moved from paper to computers 18 years ago."
State education and technology officials are also very active in sharing information and studying ways to improve data collection and presentation. Pfeiffer speaks nationwide on the topic, and Florida has hosted three national conferences in which its education department participated. "Like many things in information technology," said Florida's Lauver, "it would be a shame to reinvent the wheel."
Moving Students Forward
Looking ahead a few years, Ogle and others expect most education data collection to focus on individual students and fit into a longitudinal pattern. "This makes sense now to be linking these data," he said, "because Missouri [and other states] are offering more tests to students -- some twice a year."
The increase in testing frequency and specialization in topics are direct results of NCLB, and provide a more comprehensive snapshot of student performance at each grade level. They also produce more data.
Most states appear to be moving toward, or at least considering, LDS frameworks that allow for almost limitless data compilations.
Though often maligned by the media and school districts, NCLB has mandated data collection and analysis as a way to best educate each child. "This focus on data is meant to help the individual student," Ogle said, "which is the commitment of the NCLB program."