Towards Emotionally Healthy AI
Abstract
Artificial intelligence systems increasingly detect, interpret, and influence human emotions through recommendation, conversational companionship, and mental-health support. These systems can provide comfort, expand access to support, and facilitate human connection, but they also create risks of emotional manipulation, dependency, deceptive anthropomorphism, privacy loss, cultural bias, and inadequate accountability. We examine emotionally healthy AI through four questions: what requirements should apply across use cases; how emotional care can be distinguished from emotional control; how governance and regulation should address emotional intervention; and how AI can be used responsibly in healthcare settings. We synthesize existing governance frameworks, technical work on emotional alignment, empirical evidence on human--AI relationships, and stakeholder consultations. We organize the resulting recommendations around transparency and explainability, privacy and contextual consent, distributed accountability, risk-based governance, and technical safeguards for emotional data, non-manipulative model behavior, fairness, auditability, and interoperability. Our central position is that emotionally adaptive AI should augment human wellbeing and judgment without exploiting vulnerability, concealing simulated empathy, or encouraging unhealthy dependence.