Here we provide an example of how educational institutions can evaluate an AI tool using the nine principles and key questions of the Netherlands Commission for Unesco. We illustrate this with a fictive tool: EduSmart.

Fictive case: EduSmart (in English)

EduSmart is an adaptive Learning Management System (LMS) designed for secondary education, aimed at personalizing lessons and assignments. It offers:

  • Personalized practice exercises based on student performance.
  • AI-powered suggestions for grading and assessing written or spoken assignments.
  • A digital dashboard that allows teachers to track student progress and performance.
  • Speech recognition to support students with reading or writing difficulties, enabling oral responses.
  • For AI tools such as EduSmart, the answers to the nine key questions can often be obtained through publicly available online information and through direct contact with the tool’s developer.

1. Right to Privacy

Key question: How does EduSmart prevent the collection of personal data that is not essential to the learning objectives of the educational institution, and does it provide users with the option to opt-out of data collection entirely?

  • Score: 4/5 – EduSmart collects only essential data (performance, progress, login details) and does not request extra personal information (e.g., birthplace, family situation, full address). However, cookies are used for dashboard personalization.
  • Explanation: Users can disable certain tracking features in settings, but a full opt-out of progress data is limited, as the adaptive system relies on it.

2. Non-discrimination

Key question: What measures have been taken to ensure that EduSmart is free from biases based on gender, ethnicity, or other identity characteristics, and has there been an external audit for bias?

  • Score: 3/5 – EduSmart has trained its AI models on diverse datasets and conducted basic bias checks. External audits are not yet standard, but a pilot project is underway to involve independent reviewers.
  • Explanation: The system has been tested on English and Dutch datasets from various schools, but bias cannot be ruled out entirely.
     

3. Transparency & Explainability

Key question: To what extent can the decision-making process of EduSmart be explained, and are the models and data used transparent to both teachers and education personnel, students and parents?

  • Score: 2/5 – The system provides teachers with simple explanations (e.g., "Student X gets extra math exercises because they missed three questions"). However, the underlying machine learning algorithms are proprietary and not openly accessible.
  • Explanation: There is limited transparency regarding how AI reaches conclusions. A simplified "explainable AI" module is in development.
     

4. Accountability

Key question: Who is legally accountable for the outcomes generated by EduSmart? And how can schools or individuals challenge results gathered by EduSmart?

  • Score: 3/5 – Teachers remain responsible for final grades. EduSmart’s user agreement states that it does not bear legal responsibility for incorrect or undesired results. Schools can submit complaints to request AI corrections.
  • Explanation: Schools can contact the helpdesk for support in adjusting AI-generated insights. However, there is no formal escalation process to an independent authority.
     

5. Human Oversight and Autonomy

Key question: To what extent does EduSmart ensure that teachers retain ultimate control over its outcomes, and can teachers modify or disregard results generated by EduSmart?

  • Score: 4/5 – Teachers receive AI-based suggestions (e.g., "Average score: 7.2") but must manually enter the final grade. They can ignore AI recommendations entirely.
  • Explanation: EduSmart provides recommendations and additional exercises, but teachers can modify or disregard them as needed. No grades or assessments are automatically enforced.

6. Human Rights, Dignity, and Accessibility

Key question: How does EduSmart ensure that it is accessible to all, and that human dignity and rights are upheld?

  • Score: 3/5 – The platform includes speech support, larger fonts, and color filters for visually impaired users. However, its speech recognition struggles with speech disorders or strong accents.
  • Explanation: While inclusivity is a focus, features for students with complex disabilities are still under development.

7. Ensuring Equality in Use

Key question: To what extent can it be guaranteed that all students and teachers have equitable access to EduSmart, including the necessary infrastructure and support?

  • Score: 2/5 – EduSmart is web-based and requires a stable internet connection. Schools with poor Wi-Fi infrastructure or limited devices face disadvantages. There is no offline mode.
  • Explanation: Schools with sufficient budgets and devices can use EduSmart effectively. However, students in underprivileged areas or those with poor internet access may struggle to use it equally.

8. Ecological Impact

Key question: Are the environmental impacts of EduSmart proportionate to the educational significance of the task it performs, and have measures been taken to limit the ecological footprint?

  • Score: 3/5 – EduSmart’s AI models run partly in the cloud, using data centers powered by renewable energy. However, adaptive learning and speech recognition require significant processing power.
  • Explanation: Servers are downscaled at night to reduce energy consumption, but overall energy use remains high.

9. Training and Awareness

Key question: How much funding, expertise, resources, and time does the institution have to adequately train teachers and other personnel on the ethical, practical, and pedagogical implications of using this AI tool?

  • Score: 2/5 – EduSmart offers an online manual and webinars, but there is little in-depth training on ethical dilemmas or bias. Training mainly focuses on technical usage.
  • Explanation: Schools must largely organize their own training, which can be time-consuming. As a result, implementation is slow, and teachers are not always well-prepared for ethical concerns.

In Summary: Making A Decision

The answers to the key questions provide material for discussion. As an educational institution, what minimum scores are we willing to accept? Are certain principles more important to us than others?

Rather than relying on a single overall score, it is advisable to carefully evaluate each principle in relation to the institution’s values and priorities. In some cases, even a score of 4 out of 5 for a particular principle may not be sufficient. The Netherlands Commission for Unesco therefore encourages schools to engage in discussions on this topic, involving all relevant stakeholders— support staff, teachers, students, and, where appropriate, parents.

When considering the fictive product EduSmart, there are several key considerations that institutions should reflect on:

  1. Right to Privacy: Are we comfortable with the fact that full opt-out from data collection is not possible? Does this align with our school's privacy standards?
  2. Transparency & Explainability: Do we see the lack of an external bias audit as a major risk, or are the current internal measures sufficient
  3. Human Oversight and Autonomy: Are we satisfied with the level of control teachers retain over assessments and grading?
  4. Ensuring Equality in Use: Can our school truly accommodate all students with EduSmart—including those with limited home infrastructure or special educational needs?
  5. Ecological Impact: Do the educational benefits and reduced teacher workload outweigh the system’s higher energy consumption?
  6. Training and Awareness: Do we have the budget and time to organize our own training on ethical and pedagogical AI use?

By reflecting on these six key considerations, school leadership and teaching staff can make an informed decision on whether, and how, to implement EduSmart in their educational environment.

 

Loading...