The No Child Left Behind Act meant far-reaching changes for American schools. The act focuses on improving accountability and academic achievement, but put particular emphasis on assessment. Schools not only have to conduct assessments more often, they must demonstrate efforts to boost achievement in areas where assessments show students are failing.
But conducting statewide assessments is expensive and time-consuming -- especially when the tests include essays or short-answer questions. Although computers easily grade multiple-choice questions, essay assessments require human evaluations. This year, for example, Illinois paid $6.5 million to grade 1.2 million student writing samples, according to the Illinois State Board of Education. While federal funds are available for new tests, states must provide significant resources as well. As the number of assessments grows, states are looking to curb costs.
The Indiana Department of Education may have a solution. This year, Indiana was the first state to use computers to conduct student writing assessments, transfer those assessments electronically to a grading facility, have them graded via computer and sent electronically back to the schools. The state expects this new method to save taxpayer money, reduce testing time, speed up test results and give schools more time to help struggling students catch up.
"We've been concerned for years about the costs of state tests and the speed at which the tests get graded," said Stan Jones, Indiana's commissioner of higher education. "Right now, it takes about two months to get results back because they have to be graded by people. We've had essay tests as part of our state testing program for eight years, and we think it's an important part. Therefore we've been searching for a way to both cover costs and get our test results back more quickly."
Traditionally statewide essay assessments are graded by temporary employees who spend an average of three minutes on each one, according to Educational Testing Service (ETS). The cost of paying those employees can easily run into the millions, and the delay in obtaining results means less time to get struggling students into tutoring or remedial programs.
"The time involved is critical because these are end-of-course exams," said Jones. "If it's going to take two months to get results back, you're talking about giving the test much earlier in the year. That means less time to ensure students know the material and less time to help them if they don't."
Indiana uses ETS e-rater technology, a form of artificial intelligence designed to mimic human readers, to analyze two state-recommended Core 40 courses -- English/Language Arts and Mathematics -- for grades eight through 12. ETS and subcontractor Achievement Data Inc. provided test design and development, field test design and administration, online assessment system design, computer-based testing procedures, field test implementation, development and implementation of automated scoring procedures, score reporting, data analyses, and operational forms construction and delivery.
Approximately 60,000 Indiana students took part in the computerized assessment last spring, according to Jones. The state gave schools a choice of trying computerized testing or sticking with the traditional paper-and-pencil tests graded by people. According to Jones, 80 percent of the schools chose the online exams.
Schools that conducted online assessments received scores within three days, and the average cost per test dropped from about $30 to $15. "The primary cost saving is the human labor of having people grade the tests," said Jones. "But there are also administrative cost savings because you don't have to ship tests out and have them shipped back and reassembled."
Managing the Skeptics
Some critics of evaluating essays via computer charge that computers cannot effectively evaluate the markedly variable craft of writing.
To dispel skepticism, ETS conducted a two-year pilot program in which student essays simultaneously were graded by a computer and trained readers. The pilot found