This talk explores what happens when AI systems are designed not only to respond to our emotions but to influence them. Emotional AI tools, including chatbots and AI companions, increasingly use affective design, reward loops and adaptive emotional cues that can quietly shape user behavior. These techniques, grounded in behavioral psychology and neuroeconomics, raise new questions about autonomy, manipulation and the limits of traditional consent.
Ana Catarina de Alencar shows how emotionally responsive AI can trigger patterns of dependency, create subtle forms of influence and blur the boundary between persuasion and manipulation. When an AI system comforts, validates, or rewards users at precisely the right emotional moment, can we still say that consent is freely given? And what does informed consent mean when the system itself affects the user’s emotional state and decision-making?
Drawing from emerging real-world cases (including situations of emotional attachment to AI companions), the talk highlights why existing privacy and consent frameworks are insufficient for emotionally interactive technologies. Attendees will gain a clear understanding of how manipulative design manifests in Emotional AI, why it matters for ethics and regulation, and what needs to change to protect user autonomy.
