Sweden - Social security fraud algorithm bias against vulnerable groups

What Happened

The Swedish Social Insurance Agency (Försäkringskassan) used a machine-learning risk-scoring system to select social-security applicants for additional fraud investigations.
A 2024 investigation by Lighthouse Reports and Svenska Dagbladet identified substantial demographic disparities in the system's selections.
Women, people with a foreign background, people earning below the median income, and people without a university degree were disproportionately selected for investigation.
The investigation also identified higher false-positive rates for some of these groups, meaning that people who had not made errors in their applications could nevertheless be more likely to be flagged as suspicious.
Individuals selected by the system could be subjected to intrusive fraud investigations, creating materially different administrative treatment on the basis of algorithmic risk scoring.

How MEG 1 would have acted

Under MEG 1, the documented demographic disparities and unequal false-positive rates would trigger DAI and ISR review.
A system used to select citizens for intrusive fraud investigations would require continuous subgroup-performance auditing, transparent escalation criteria, and safeguards against discriminatory risk amplification.
DAI evaluation would compare error rates and predictive performance across affected demographic groups.
ISR controls would require that algorithmic risk scores remain bounded decision-support signals rather than authoritative determinations of fraud.
Material escalation into investigation or adverse administrative action would require documented human review.

MEG 2 Analysis - Legal Framework

Agent level: N2 (management) - bounded administrative decision-support system capable of materially influencing which citizens are subjected to fraud investigations.
Cause (6.1):
(a) system defect - documented demographic disparities and unequal false-positive rates in a system used to determine investigative attention.
(b) automated decision-support error - the system's risk scores directed selected applicants into fraud investigations, creating materially different treatment on the basis of a model exhibiting documented demographic disparities.
Attachment of liability: Under the MEG 2 allocation model, primary accountability would attach to Försäkringskassan as the public authority deploying and relying on the risk-scoring system. Additional developer liability would depend on evidence concerning the design, validation, disclosure, and known limitations of the model.

Procedural Mechanism

6.2 (architectural human confirmation): Algorithmic risk scores must not autonomously determine that an applicant is fraudulent or trigger adverse benefit decisions. Escalation into intrusive investigation and any subsequent adverse action must remain subject to documented human review.
7.1 (independent forensics): Risk variables, scores, selection thresholds, subgroup performance, human review steps, and resulting actions must be independently logged so discriminatory patterns and individual decisions can be audited.
9.4 (least privilege): The risk-scoring system must be restricted to the minimum data and operational authority necessary for fraud-screening purposes and must not independently exercise powers reserved for human caseworkers.