The most common failure pattern
in edtech purchasing is buying a solution before clearly defining the problem.
A vendor demonstration is designed to impress it. It shows the tool working
well, in favorable conditions, on favorable tasks. What it does not show is
whether those tasks are the ones your teachers and students actually need help
with.
Before any vendor presentation,
convene the leadership team around a single question: what specific
instructional or operational challenge are we trying to address? The answer
should be concrete enough that you could measure whether a tool actually helps
with it. Vague answers like 'improve personalized learning' or 'save teacher
time' are not problem statements. They are marketing language. If the tool's
pitch doesn't directly address the specific challenge your district faces, that
misalignment is worth naming early.
The National Association of
Elementary School Principals, in its guidance for school leaders evaluating AI
tools, frames this as the non-negotiable first step: clarity about the problem
prevents wasted dollars and frustration later.2
This is the question most
standard edtech purchasing processes do not ask, and it is the one that
distinguishes AI tools from everything that preceded them. Traditional edtech
collects structured data: correct answers, time-on-task, completion rates. AI-powered
tools collect unstructured input: what students write, how they phrase
questions, what paths through material they take and abandon.
The Future of Privacy Forum's
guidance on vetting AI tools in schools identifies the model training question
as the critical gap in most district reviews: will student-generated content be
used to train or improve the underlying AI model?3 Some vendors explicitly prohibit this. Others are vague. If a student's essay
or tutoring session feeds back into a model, that data may persist indefinitely
and cannot be fully removed later. Ask this question directly, in writing, and
treat a vague or evasive answer as a significant concern.
A vendor that cannot tell you
clearly whether it uses student inputs for model training is a vendor you
cannot evaluate for data privacy compliance, regardless of what its FERPA
boilerplate says.
Every AI edtech vendor will
provide testimonials, case studies, and internal research. None of that is
independent evidence. What you are looking for is third-party studies, ideally
peer-reviewed, that evaluated the tool's impact on student outcomes in contexts
comparable to yours. Similar grade levels, similar student populations, similar
implementation conditions.
This bar is harder to meet than
it sounds. A 2024 analysis found that only 32% of edtech tools met any of the
Every Student Succeeds Act evidence tiers, rising to 45% by 2025 and remaining
a minority of the market.4 That means the majority of edtech products on the market, AI-powered or
otherwise, do not have strong independent evidence of effectiveness.
'Research-based' in a vendor pitch often means the vendor conducted or
commissioned the research. That is not the same thing.
Asking for independent evidence
is not a reason to reject every tool without a randomized controlled trial. It
is a way to calibrate how much confidence you should have before committing to
a full deployment and to build the right expectation that a pilot comes first.
One of the clearest patterns in
edtech implementation research is that top-down rollouts fail at much higher
rates than teacher-led ones. A tool that is approved at the board level and
handed to classrooms without teacher input will encounter resistance,
workarounds, and eventually disuse. The 65% unused license figure is not
primarily a purchasing problem. It is an implementation problem, and it starts
at the approval stage.
Before approving any AI tool,
at least some of the teachers who will use it should have tried it. Their
feedback is not a formality. It is the most reliable signal available about
whether the tool fits into actual classroom practice, whether students engage
with it, and whether it solves the problem or creates new ones. The NAESP
guidance is direct on this point: teacher-led pilots create champions who model
use for peers; top-down rollouts breed resistance.2
A six-week pilot with
structured teacher feedback and usage data is not a delay. It is the minimum
information a board needs to make a responsible approval decision on a tool
that will interact with every student in the affected classrooms.
Many districts purchased edtech tools with pandemic-era stimulus funding that has since dried up. The sustainability question is not hypothetical: can the district sustain the licensing cost, the required professional development, and the ongoing vendor relationship once initial funding ends?
For AI tools specifically, sustainability includes a question that does not apply to static software: who is responsible for monitoring the tool as the underlying model changes? AI vendors update their models, sometimes significantly. A tool approved in September may behave differently in March because the vendor retrained its model. The district needs to know who is responsible for reviewing tool performance after major updates, and whether the vendor will notify the district when updates occur that could affect student interactions or data handling.
Assigning ownership of each approved AI tool to a named staff member, with a defined review schedule, is the single most practical governance step a board can take when approving a new AI product. Tools without owners tend to drift into the unused license category. Tools with owners get used, evaluated, and either improved or replaced.
These five questions will not make every AI approval decision obvious. But they will make the decision a real one, based on your district's actual needs and circumstances, rather than a vendor's demonstration of what the tool can do under ideal conditions. That is the standard your students deserve.
