Algorithms are quietly deciding which children receive support and which do not, often without parents or teachers knowing exactly how the software works. As local authorities turn to technology to manage rising caseloads, the humans in charge frequently lack the training to oversee the automated systems they are purchasing.
AI tools are automating high-stakes decisions for children with special educational needs and disabilities (SEND), but a lack of local oversight and technical training means these choices often happen without clear accountability or privacy protections.
If your child is in the SEND system, an algorithm may already be the invisible hand moving them up or down a waitlist or determining their level of resource allocation. When technology outpaces the training of local officials, the "human in the loop" you speak with at the council may not actually understand why a tool flagged your child’s file or denied a specific request. This creates a "black box" where life-altering decisions for vulnerable children are made by software that neither the parent nor the caseworker fully understands.
Local authorities are under immense pressure to cut costs and clear backlogs in special education services. Researchers interviewed 17 UK policymakers and professionals to map the gap between national "pro-innovation" policies and the reality of local implementation. They found that while the government pushes for AI adoption, the local staff responsible for these tools are underfunded and technically overwhelmed, leading to a breakdown in responsible oversight.
Staff are increasingly relying on unauthorized "shadow AI" to keep up with heavy workloads, creating significant risks for child data privacy. The study identified several specific failure points in the current system:
- Market-government asymmetry. Private tech vendors often hold all the technical knowledge, leaving public officials unable to properly vet the tools they buy or challenge the results they produce.
- Vanishing accountability. It is becoming increasingly difficult to pinpoint who is responsible for a bad decision when an algorithm is the primary driver of the assessment.
- Workforce unreadiness. Most local authorities lack standardized metrics to test if an AI tool is biased against certain groups of children or if it is providing accurate recommendations.
- The "Fault Line." There is a total disconnect between high-level ethical guidelines written in London and the practical, day-to-day reality of managing a SEND department with limited resources.
The "efficiency" promised by AI is frequently a shield for austerity. By offloading resource-gatekeeping to software, local authorities can distance themselves from the social and political fallout of denying support. If a computer generates a "priority score" that leaves a child without a 1-on-1 aide, it is much harder for a parent to argue against a seemingly objective, data-driven result than it is to challenge a human caseworker’s opinion.
The study relies on a very small sample size of 17 professionals, meaning it captures the culture and concerns of the workforce rather than hard data on how specific AI models perform. This is a qualitative preprint that has not yet undergone formal peer review. Additionally, the findings are specific to the UK’s legal and administrative framework for SEND services and may not directly reflect how school districts in the U.S. or elsewhere manage similar assessments.
- If your child is undergoing a SEND assessment or annual review, ask your local caseworker specifically if any automated tools or algorithms are used to rank your child’s priority or calculate their resource allocation.
- If a support request is denied based on a "score" or "assessment tool," insist on a written explanation of how a human reviewed the evidence and whether they had the power to override the computer's recommendation.
- If you are concerned about your child's data, request the council's Data Protection Impact Assessment (DPIA) for any third-party AI platforms they use to process sensitive student information.
AI is no longer a future prospect for education services; it is a current administrative tool that lacks the guardrails necessary for high-stakes child welfare. Parents must act as the primary "human in the loop," demanding transparency about the software programs that are increasingly acting as gatekeepers to their children's support.
Sitong Lyu, Shabnam Taghiyeva, Mohit Kukadia et al. (2026). Fault Lines: Navigating Ethics and Responsible AI Where National Policy Meets Local Practice in Public Sector Transformation. arXiv (preprint). — arxiv.org