Racial Bias Lawsuit Against AI in Ontario Prisons

What happened?

In February 2025, the law firm Koskie Minsky filed a class action lawsuit against the Ontario government alleging that the SAFER (Security Assessment for Evaluating Risk) artificial intelligence system, used since 2021 in provincial prisons to classify inmates according to risk level, disproportionately assigns Black individuals to maximum security regimes. The investigation by journalist Desmond Cole, published by The Breach, revealed that between 2022 and 2025, Black people—who make up approximately 5% of Ontario's population—represented nearly 27% of inmates in maximum security, while white people are underrepresented in that category. The analysis of government data was conducted by criminologist Scot Wortley of the University of Toronto.

SAFER was designed by researcher Grant Duwe, who did not respond to requests for comment. The tool assigns a risk score based on factors such as arrest history and prison discipline. However, these data reflect decades of racial discrimination in policing and judicial processes, leading to systemic algorithmic bias.

Why is it important?

This case highlights how artificial intelligence systems can perpetuate and amplify existing racial biases in the criminal justice system. SAFER relies on historical data that, according to internal training documents cited by The Breach, the government itself acknowledged would 'likely contribute to the overrepresentation' of Indigenous and racialized people in maximum security. However, while mitigation measures were adopted for Indigenous inmates—such as considering contextual factors—similar protections were not extended to Black inmates. The lawsuit alleges this constitutes illegal racial discrimination under the Canadian Charter of Rights and Freedoms.

The case joins other global incidents where AI has shown racial bias, such as the COMPAS system in the United States, used to predict recidivism, which overestimated recidivism risk for Black people. It also recalls the bias in Amazon's hiring algorithms, which penalized women's resumes. The key difference here is that SAFER is a government tool in a context of deprivation of liberty, where consequences are particularly severe: inmates in maximum security have less freedom of movement, less access to programs and visits, and harsher conditions.

What consequences will it have?

The class action seeks to have the use of SAFER declared illegal and to compensate those affected. If successful, it could set an important precedent regarding the responsibility of public administrations when implementing AI tools in sensitive contexts. It could also force a review not only of SAFER but of other similar systems used in Canada and other countries. Ontario's Ombudsman has received complaints about SAFER's disproportionate impact, and a ministerial review of the program is underway.

More broadly, this case reinforces the need for independent algorithmic bias audits and transparency in algorithms used by the public sector. The European Union is already advancing with the AI Act, which requires impact assessments for high-risk systems like this. In Canada, the Artificial Intelligence and Data Act (AIDA) is still in the legislative process, but cases like this could accelerate its implementation.

For technology companies, the case underscores the importance of designing algorithms with fairness from the start, not as an afterthought. For users, especially racialized communities, it is a wake-up call about how technology can reinforce historical injustices if left unchecked.

What should readers know?

• SAFER was designed by Grant Duwe, who did not respond to requests for comment. Duwe is known for developing risk assessment tools in Minnesota's prison system.
• The lawsuit was filed in February 2025 and is in the certification phase as a class action. It is based on government data analyzed by Scot Wortley, an expert on racial discrimination in the justice system.
• Ontario's Ministry of the Attorney General has not responded to questions about SAFER's methodology or accuracy. It has also not provided details on mitigation measures implemented for Indigenous inmates.
• Ontario's Ombudsman has received complaints, and a ministerial review of the program is underway. Additionally, the lawsuit cites internal documents showing the government was aware of bias risks.
• This case joins other global incidents: the COMPAS system in the U.S., the benefits algorithm in the Netherlands that discriminated against ethnic minorities, and the predictive policing system in the UK that also showed racial biases.

This case highlights that artificial intelligence is not neutral: if fed biased data, it reproduces and amplifies existing injustices. Transparency and fairness must be prerequisites for its implementation in the public sector. As the lawsuit stated, 'the government cannot outsource discrimination to an algorithm.'