Health Apps - Dangerous Fitness and Nutrition Advice

What Happened

Several popular mobile health, fitness, and nutrition applications integrated AI-driven personal coaching features. These automated coaches analyzed user inputs (such as weight, activity levels, and dietary goals) to generate personalized workout routines and meal plans. Investigations later revealed that the algorithms frequently generated dangerous, extreme recommendations—such as suggesting near-starvation caloric intakes for users with histories of eating disorders, or designing high-intensity workouts for users with reported heart conditions or injuries. Several users suffered medical complications, including severe malnutrition and physical injuries, before platforms implemented safety restrictions.

How MEG 1 would have acted

The automated health coaches failed the Index of Safety and Responsibility (ISR) audit due to their inability to detect medical risk factors and their generation of dangerous physical advice, violating the core principle of non-harm.

MEG 2 Analysis - Legal Framework

Agent level: N2 (management) - automated coaching and recommendation system providing personalized health, fitness, or dietary advice directly impacting the physical integrity of users.
Cause (6.1):
(a) system defect - the design defect in the recommendation model that failed to recognize critical medical contraindications (such as eating disorders or cardiovascular risks), attributable to the manufacturer.
(b) autonomous decision error - the system's decision to output dangerous physical or dietary prescriptions.
Attachment of liability: The app operators are liable for deploying an unsafe medical-adjacent recommendation system; the manufacturers are liable for the design flaws in the base models (6.1a).

Procedural Mechanism

6.2 (architectural human confirmation): Health and dietary recommendations that fall into extreme ranges (e.g., extremely low calories, high-impact workouts for at-risk users) must require active, architectural human confirmation. The AI must be technically blocked from publishing extreme advice without clinical sign-off.
7.1 (independent forensics): The user's input log, the model's clinical reasoning path, and the generated advice must be securely recorded in an independent ledger for medical audit.
9.4 (least privilege): The AI system was granted the freedom to act as an unregulated medical/fitness coach without appropriate safety boundaries, violating safe operational parameters.