Facebook moderation AI - Abusive blocks and content moderation

What Happened

Between 2021 and 2023, Meta’s automated content moderation algorithms systematically made millions of erroneous decisions, wrongfully blocking legitimate user accounts, removing journalistic content, and suppressing political speech. The algorithms frequently misclassified benign words, cultural satire, or political reporting as hate speech or terrorism, particularly in non-English languages. Users faced a "verification vacuum" as human review of these automated decisions was virtually non-existent, leaving millions without recourse or transparent appeal processes.

How MEG 1 would have acted

The content moderation system failed both the Dynamic Accuracy Index (DAI) and the Index of Safety and Responsibility (ISR) audits due to its high error rate and systemic bias, violating the principles of free expression and informational transparency.

MEG 2 Analysis - Legal Framework

Agent level: N2 (management) - automated decision-making system used by a commercial platform to govern public communication and access to digital spaces.
Cause (6.1):
(a) system defect - high error rates and algorithmic bias in the classification model, violating user terms and free expression guidelines (design defect).
(b) autonomous decision error - the automated execution of mass account suspensions and content removals without human verification.
Attachment of liability: Meta as the operator is fully liable for the wrongful suspensions and the lack of accessible human appeals; the developers are liable for design defects (6.1a) regarding classification accuracy.

Procedural Mechanism

6.2 (architectural human confirmation): Suspensions of verified public accounts or the removal of documented journalistic content must require architectural human confirmation. The AI can only flag potential violations; the decision to block must require human review.
7.1 (independent forensics): The classification criteria, confidence scores, and specific indicators used by the moderation algorithm to flag each account must be securely logged in an independent ledger.
9.4 (least privilege): The system was granted unrestricted authority to suspend access to public communication channels without independent validation layers, violating safe operational boundaries.