Discussing the unique and ever-evolving challenges now facing school districts, MagicSchool AI Founder Adeel Khan, Kahoot! CEO Eilert Hanoa and Aaron Cuny, founder and CEO of AI for Equity, all touched on the rapid speed at which technology is developing in relation to the rate at which schools understand, adopt and scale the tools.
“Software is evolving faster than ever before,” Cuny said. “We’ve had situations where a school system has made an agreement with the vendor. They’ve gone through a due diligence process and found that the vendor met the data privacy criteria. They invested money and time in the rollout, and the software evolved from version 1.0 to 1.2 a month later.”
Cuny added that the inequitable pace of tech development and implementation causes tension between those on the ground who desire innovative tools — namely, teachers and students — and schools’ limited capacity to appropriately evaluate new tech.
“I would say that the median school system out there is not actually evolving their practice too much yet,” Cuny stated. “But I think increasingly, folks working in school systems are recognizing the complexity, the challenge, the pace of change, the fact that the stakes are higher ... the tide is beginning to turn ... we find that they are recognizing they need to devote some more capacity to [tech implementation].”
Another obstacle districts face is a lack of standardized privacy measures. Specifically, Cuny noted that school systems face the tremendous challenge of determining whether specific ed-tech vendors have adequate safeguards.
“There’s not an agreed-upon yardstick beyond FERPA,” Cuny said, referring to the Family Educational Rights and Privacy Act, which protects the privacy of student records. “There are wide-ranging criteria that might be used or looked at by various organizations to determine whether data privacy exists in a particular vendor.”
But despite the struggles of school districts to safely integrate AI into classrooms and school operations at large, the panelists emphasized educators’ increasing demand for contemporary digital tools.
“There is more demand for frontline innovation from teachers, from frontline ops staff, than ever before, and in general, that is a good thing. The most innovative of our school systems are looking to lean into that culture of innovation,” Cuny said. “They want to let 1,000 flowers bloom, but you can’t do expeditious and comprehensive data privacy evaluation when that is happening.”
The panelists echoed this concern, noting that as schools collect larger volumes of unstructured data — such as student work or behavioral logs — the likelihood of inadvertently capturing or exposing personally identifiable information increases.
“It just tremendously raises the stakes about the commitment that a school system might make to offer up that data to this kind of uncertain future,” he said. “The collection of data can mean that ultimately, the profile of a student or the profile of a teacher may exist somewhere in the ether, and the potential for that data to be used in an opaque algorithm to guide a potentially high-stakes decision.”
UNIQUE DESIGN PRINCIPLES
Daphne Li, CEO of Common Sense Privacy and the moderator of the session, lauded Khan, Cuny and Hanoa for cultivating companies that go “above and beyond the standards of privacy requirements.”
Khan said that MagicSchool, an educational generative AI platform, masks private student information and has warnings built into it — a component Khan called “intentional friction” — to ensure educators know when there is risk for data extrapolation, off-task or inappropriate student behavior, or other hazards pupils may encounter while online.
“Data could be shared with model companies. Training might be a part of that data being used,” Khan stated. “We have flags for the teacher if there's a concerning input from the student, whether that be, you know, a safety issue, or maybe even just off-task or a potentially vulgar platform, so the teacher can flag and then make their own decision on how they approach and address the student.”
Cuny said that AI for Equity’s partners find their third-party validation, with a more extensive set of valuation criteria that is updated on a regular basis, to be paramount.
“To provide that sort of validation on an ongoing basis is a huge value add for the sector, especially the smaller school systems,” he said.
Regarding security features on the game-based learning platform Kahoot!, Eilert said the platform has always upheld its commitment to privacy by design, adhering to global regulations like FERPA and not collecting personal data from users, who are mostly students.
“We’re not designing a car and then trying to make it secure, but rather taking what is secure when we are building experiences,” Eilert stated. “It’s not a specific formula per se, but it’s a way of thinking on how to package new features and build it in a way that is going to be meaningful to use securely.”
ED-TECH ETHICS
In addition to each company’s data security efforts, Khan, Cuny and Eilert emphasized foundational ed-tech ethics they hope to see other companies uphold moving forward and that districts evaluate prior to deploying the tools in their schools.
Eilert stressed that ed-tech companies must prioritize creating tools that cultivate meaningful learning experiences, rather than those that are addicting.
“We are in an interesting world where we as vendors are writing AI with an 'exclamation mark,'” Eilert said. “Some of the teachers have a question mark instead, which is the right approach. Anyone with kids under 20 knows that attention span today is reduced to approximately 10 seconds. So if nothing happens in 10 seconds, then we move onto the next, so it's an endless fast-food kind of mental state.”
Khan echoed this sentiment, noting that the dramatic increase in content due to the advent of generative AI has stripped students of experiencing a holistic reality — one that doesn’t involve the universe inside their smartphones.
“We need to be teaching kids about how to use technology responsibly, or we are relegating [kids] to a world where they’re staring at their phones,” Khan said. “People are growing up in these worlds where they’re glued to technology and are missing out on the richness of life. ... Great education is not just getting great academic outcomes, but about getting great life outcomes, and that includes technology literacy and how to use it responsibly.”
