The paper, The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis, reviewed findings from 51 research studies published between November 2022 and February 2025. Authors Jin Wang and Wenxiang Fan, both from Hangzhou Normal University in China, aimed to assess the effectiveness of ChatGPT in improving students’ learning performance, learning perception and higher-order thinking. They concluded that ChatGPT use was associated with improved student learning performance, stronger engagement and motivation, and moderate gains in high-order thinking skills. They also argued that tools were most effective when used in structured educational settings, such as problem-based learning, and when implemented consistently for four to eight weeks.
“Based on these results, governments could introduce policies to promote the integration of ChatGPT into education, encouraging universities and secondary schools to apply ChatGPT appropriately,” the paper’s conclusion reads. “Additionally, ChatGPT could be actively incorporated into STEM and related courses, skills and competencies development training programs, and language learning and academic writing courses to enhance student learning.”
The paper was published in May 2025 and retracted April 22, 2026. The retraction notice did not cite specific issues, only “concerns regarding discrepancies in the meta-analysis.”
Since its initial publication a year ago, the paper had received unusually broad attention for an education meta-analysis, with extensive coverage in AI- and education-focused media. For example, the paper made Forbes’ shortlist of AI research papers everyone should read. Nature shows it has been accessed 492,000 times, cited 270 times and achieved an Altmetric score of 1025, ranking first in online attention among articles of a similar age from the same journal.
However, the retraction was not the first sign of skepticism. Finnish researcher Ilkka Tuomi shared qualms on LinkedIn 11 months ago, criticizing the lack of quality assessment for the 51 research papers and the small sample size for experimental groups — 33 of the analyzed studies had less than 35 students in the group using ChatGPT.
“The even bigger issue really is that this paper got a lot of attention at the time,” Ben Williamson, lecturer at the University of Edinburgh, wrote on LinkedIn this week.
In response to the retraction, Jeff Walsh, founder of a student-focused Internet search platform, said on LinkedIn that teachers already knew to be skeptical of the results.
“If you’re an educator of some kind, I tip my hat to you,” he wrote. “If you’re building in ed tech, don’t just build something you think people may want. We have the added burden of building something that improves learning outcomes or we return their money.”
Williamson agreed.
“Finally we’re seeing some recognition that rushed evidence about AI in education was wrong,” he wrote.
The Center for Digital Education has reached out to Nature Humanities and Social Sciences Communications for comment and will update this story if they respond.
