Japan - ChatGPT-like AI in Administration Generating Erroneous Data
What Happened
Several local municipal governments in Japan deployed localized generative AI chatbots (based on ChatGPT APIs) to assist public officials in drafting administrative documents, answering citizen inquiries, and synthesizing public policy data. It was later discovered that the AI systems frequently "hallucinated" non-existent local ordinances, generated incorrect statistical data about public finances, and provided citizens with erroneous information regarding tax exemptions and welfare eligibility. In several instances, public officials included these fabricated figures in official municipal reports before the errors were caught, leading to administrative confusion and the suspension of the automated tools.
How MEG 1 would have acted
The generative models failed the Dynamic Accuracy Index (DAI) audit by presenting fabricated administrative facts as official data, directly violating the cognitive integrity and informational self-determination of both public officials and citizens.
MEG 2 Analysis - Legal Framework
Agent level: N2 (management) - generative assistant integrated into public administration workflows, acting as a representative of municipal authority and policy drafting.
Cause (6.1):
(a) system defect (hallucination) - the foundational model's design defect that allows it to generate plausible-sounding falsehoods as verified public policy facts, attributable to the model manufacturer.
(b) autonomous decision error - the system's decision to output fabricated administrative data without verification.
Attachment of liability: The municipal operators are liable for failing to implement verification layers; the model manufacturers are liable for the model's design defect (6.1a) regarding information reliability.
Procedural Mechanism
6.2 (architectural human confirmation): Incorporating AI-generated outputs into official public reports, laws, or municipal responses must require strict, architectural human confirmation. The AI can only act as a draft editor; a human officer must verify every data point and sign off before publication.
7.1 (independent forensics): Every session log, prompt input, and model output must be preserved in a secure, independent ledger to allow for administrative auditing.
9.4 (least privilege): The generative system was granted direct access to administrative drafting pipelines without real-time factual checking layers, violating safe operational boundaries.