When President Bush signed the No Child Left Behind Act into law Jan. 8, 2002, states suddenly faced a huge technology problem -- not only were they required to collect and report school performance data, they also had to disaggregate that data to track the performance of individual students in various ways. For most schools, that meant a level of tracking and reporting far beyond their IT capabilities.
A year later, many states still struggle with this task. But a few are well ahead of the pack.
Educational Performance and Information
Michigan's Center for Educational Performance and Information (CEPI) is responsible for collecting, managing and reporting public school data throughout the state. Gov. John Engler created CEPI by executive order in September 2000.
"The executive order was the starting point for us in terms of the governor providing executive leadership that the collection of educational data was going to be important," said Andrew Henry, CEPI's director.
CEPI collects discrete student-level data, including age, date of birth, ethnic denomination, test scores and programmatic information -- whether a student participates in state education programs such as special education or Title I, or is considered migrant or homeless.
Michigan uses the data to answer specific questions about its schools and hold them accountable for every student's success or failure.
The benefits of answering such questions and tracking overall school performance may be obvious, but the process of collecting that data is much more complex. Technology has made gradual inroads in education, resulting in stovepipes of information separated by incompatible technologies and data structures. Getting all that data to collaborate seamlessly to meet No Child Left Behind requirements appears a Herculean task.
"Collection of data is a very complex issue," Henry said. "One of the most important tools we have is our metadata dictionary, which is our definition of what we're going to collect. That's probably the most important step, because if you don't know what you're going to collect, you're in big trouble. Also, if you're not publishing how that data has to be pushed to the state, you're in really big trouble."
CEPI worked with various state agencies, including the Department of Education, the Department of Career Development, the Department of Treasury, and local school associations and districts to come to a consensus on data definitions. The organization also worked with vendors and various state agencies to determine how local school districts' existing systems could interface with the state.
"Rather than re-create or reinvent a system that's already working, we work with them and develop applications in partnership," said Henry. "It's going down to the grass roots and saying 'What have you developed, and what is going to make this job easier because this is a very difficult task?' Any success we have is a tribute to the local school districts entering or abstracting data from their systems. I think a big win is going to these people who are developing systems and asking them, 'How can we tap into the expertise you have?'"
CEPI also has worked with the U.S. Open e-Learning Consortium, a group of software companies and school systems that created the Schools Interoperability Framework (SIF) in 1999. SIF is a set of rules and definitions that allow software programs from different companies to share information quickly, seamlessly and securely. Using XML-based software that conforms to SIF allows schools and school systems to share data without any additional programming. SIF could make meeting requirements of the No Child Left Behind Act much easier.
"Right now, we essentially collect flat files, but we want to go to native XML, and SIF is an integral part of that," said Henry.
CEPI also collects teacher data. That information allows the state to examine teacher preparation, professional development, teacher quality and other criteria, giving a good picture of teacher performance and where improvements can be made.
Accountability for Academic Progress
Indiana also is making significant strides in tracking and reporting school performance data. The state's Accountability System for Academic Progress (ASAP) provides academic performance data for all Indiana schools via the Web and allows users to search for data in a variety of ways.
"We've always had a lot of data in the department," said Suellen Reed, Indiana's state superintendent of instruction. "We collect data on everything, but it's never been very user-friendly. If you were technologically talented you could dig out almost anything you wanted to know, but the average person couldn't do that without a lot of help."
ASAP went live April 15, 2002. Using in-house IT talent, the state created a user-friendly format that allows teachers, administrators, students, parents or businesspeople to analyze school data. Areas of weakness can be compared year to year. ASAP also provides the variety of data needed to facilitate school compliance with No Child Left Behind requirements.
"It's a continuous improvement model that looks at students over time," said Gary Wallyn, director of school data reporting. "The student-level data will give us much more precise information about student improvement, mobility, graduation rates, etc. Down the road we'll hopefully integrate XML technology so it will be a much quicker system, and we'll be able to harvest data."
"The schools are now analyzing data better because they have a tool that facilitates that," said Reed.
Making the Transition
Once all this data is collected, how will states make the transition from gathering data to improving schools?
"We aren't the agency that's going to say a school is not making adequate yearly progress," said Henry. "But we are very much charged with collecting and reporting data in a way that's useful to the Department of Education that's charged with making that determination. Every system they're developing now that requires educational data is in conjunction with CEPI. We're right there in the thick of it in terms of providing the data that will be used to determine whether a school is measuring up to Department of Education standards."
Indiana officials are hoping schools will evaluate their own data and play a larger part in monitoring their own areas of weakness. "Everybody is searching for more time," Reed said. "Schools prior to this had to spend a lot of time collecting data, looking at data, putting it in graphical format. Hopefully this will save them time and allow them to monitor their own progress toward improvement."
With or without the No Child Left Behind requirements, Henry said analyzing data at the individual student level has already been extremely valuable to Michigan.
"The state will now really reveal how a school is performing," he said. "An aggregate number tends to shield us from the particulars. You can have a school that's performing very well in the aggregate, but if you start to pull it apart, you reveal places where the school needs work."