Presenting their firsthand experience with the AI-powered data analytics platform Doowii as a case study, officials from Florida Virtual School told attendees at the Future of Education Technology Conference on Monday that focus on the end-user experience is paramount at every step.
Melody Deetz, senior director of learning systems engineering at Florida Virtual School, said meeting users where they are makes the difference between giving teachers a new tool and a new chore.
“Data ends up being very, very important for teachers if they can ask for it in a way that makes sense to them, in their workflow, and they can get it easily and in real time,” she said.
Doowii brands itself as a conversational analytics tool which, once integrated with a learning management system, invites users to ask questions in their own words that might be answered by data, like, “What are the bottom 10 performing standards in geometry?” or “What’s the activity count for online students at each hour of the day?” After asking clarifying questions, the tool then generates a graph with a written summary, methodology, assumptions and interpretations.
Doowii’s founder and Head of Customer Experience Matthew Young broke down the company’s collaborative process with districts into four steps, starting with organizational readiness, which he said depends upon leaders with a vision who encourage curiosity and experimentation.
Martin Kelly, senior director of curriculum development at Florida Virtual School, said the next step — defining goals — is about identifying clear, quick wins that could demonstrate the value of the project.
Step three, Deetz explained, is creating a data inventory.
“We needed to make sure we were all speaking the same language, and we had a shared understanding of what data we were going to provide to Doowii,” she said. “So for example, enrollment means one thing in the full-time program, and it might mean something different in one of the franchises, so [it was about] identifying what were the outcomes that we wanted to provide to teachers, and what is the data that would support those outcomes?”
Young said once IT staff and district leaders have done all this — shared a vision, set up change management processes, established goals and completed data inventories — they’re probably ready to move into the execution stage.
“From what I’ve seen, a lot of teams typically want to move into this really quickly, without building the proper momentum and the proper foundation to have a successful execution and implementation,” he said.
It’s been less than a year since Florida Virtual School moved into that phase, and Young said his experience with it and other districts has clarified some lessons. One is that everyone’s data is likely to have issues, whether it be system incompatibilities, inconsistent IDs across platforms, missing or incomplete or duplicate records, old data that doesn’t match modern structure, or practice data mixed with real data. Another is that teachers and staff may need more support and training than anticipated, whether for navigating the platform, AI literacy, prompt engineering, or interpreting and using the data.
Perhaps most importantly, though, is that the system must be designed for the experience of users.
“I think one of the things we learned was, Doowii wasn’t in the right spot. We had the data, we were able to respond to their questions, but it required them to jump into a completely different system that they weren’t used to,” Young said. “Our lesson was, bring the solution all the way to them and insert it into their workflow.”
Kelly translated the lesson as: Spend time with your users.
“Spend time with your teachers. Spend time with your student success teams or professional development teams, whoever might be your end user for your big project,” he said. “Spend time with them and understand what they’re struggling with, what they care about, what success looks like for them, and then move forward with your planning and into your execution.”
