the e-rater generated grades virtually identical to those of the readers. "The technology is very good at analyzing the same things humans are trained to analyze within student essays," said Richard Swartz, ETS's executive director of technology products and services. "It gives you almost exactly the same results without the time and money requirements."
Because e-rater is designed to mimic human evaluators, it does not necessarily improve on a human evaluator's performance, nor was it designed to.
"If you're comparing it to a good high-school English teacher, there's no question the teacher can grade it better," said Jones. "But on state tests, the people hired are comparing students to each other, so it's more of a sorting process. So while this isn't a good alternative compared to a high-school English teacher, it is a good alternative for testing."
Other states are waiting to see the results of Indiana's pilot. Swartz predicts more states will adopt computer grading for essay tests, as long as Indiana's experience remains positive. "As computers continue to become less expensive and more powerful, there will be more of this testing," he said. "It's going to be the wave of the future."
In June, the Tennessee Department of Education said it intends for all public-school students to take their mandatory exams on computer by 2010. The state will try it in 10 districts during the next school year and expand the practice if things go well.
Computerized testing requires state schools to reach a certain level of technical proficiency, and some still have a long way to go. States must ensure all school districts have the platform to support testing, enough computers for all students to take the tests and security adequate to protect systems from being compromised. Indiana has one of the highest computer-to-student ratios in the country -- with one instructional computer for every three students -- but many other states are struggling. Some already tried computerized assessments only to abandon the plan because they couldn't obtain computers for all students.
Once a state gets its schools properly equipped, however, computerized testing requires very little from the teacher's perspective. "We have training sessions for the coordinators in each school for the administration of it, but for teachers and students, it's ready to go," said Swartz.
So far, Indiana is pleased with the pilot's results. "It's working well," said Jones. "Teachers like that they get the scores back quickly, and students like taking the test online. We've been surprised that the schools have so readily adapted to this. We anticipated there would be more skepticism. Clearly that skepticism is out there, but it's more than outweighed by the benefits."
Jones said Indiana may look next at computerizing biology tests. The state also is working with universities, hoping that exams conducted at the 11th- and 12th-grade levels eventually can double as college placement exams and be electronically transferred to colleges the students plan to attend.
For now, the state simply hopes to reduce testing costs and translate faster test results into improved student achievement. "Clearly in terms of state testing, time and money are important," said Jones. "But the other thing is that when teachers have to wait too long to get results back, the results become significantly less important. If they can get results back in three days, those results become part of their teaching and they can adjust their strategy to help students."