“Some of the potential risks come from when AI chatbots are trying to be helpful… [They might say] with certainty, like, ‘You’re going to definitely be okay,’ but that’s not actually therapeutic.” – Marlynn Wei
“Where would the line be between acting like something’s your friend versus getting therapeutic support? That’s a challenge these general purpose tools are facing, both [when it comes to] ‘How do you actually build it?’ and ‘How do you test for safety?’ And also for policymakers, because how do you draw a line for tools that theoretically could be used for multiple purposes?” – Miranda Bogen
“We have an opinion on what we think a current best practice might be, but we also acknowledge that because of this technology…usage is evolving so fast, the evaluation has to evolve with them.” – Luca Belli
Description
While many people have found benefit and respite in using chatbots for companionship, mental health, and emotional support, the widespread adoption of these tools has also resulted in harm and raised deep concerns about identity and safety. How are chatbots shaping people’s understanding of themselves? What concerns do therapists have about their use? How might these tools be designed and implemented to prioritize users’ wellbeing? What kinds of guardrails, regulations, and safety protocols might be effective?
In connection with Data & Society’s ongoing research on mental health and chatbots, on February 26 we explored these questions and more in a conversation moderated by researchers Livia Garofalo and Briana Vecchione. Together with Luca Belli, AI safety lead at Spring Health; Miranda Bogen, founding director of the AI Governance Lab at the Center for Democracy & Technology; and psychiatrist and psychotherapist Marlynn Wei, they discussed the profound shifts in how people seek help and support, and how mental health professionals, policymakers, and tech designers are navigating these shifts now.
Speakers
Luca Belli is the policy person in the tech discussion and the tech person in the policy one. He builds bridges between research, engineering, product, policy, and public interest outcomes. His work focuses on how algorithms shape culture and behavior, from co-founding Twitter’s machine learning ethics, transparency, and accountability team to auditing systems that reveal hidden assumptions in digital design. Belli is currently the AI safety lead at Spring Health and a non-resident research fellow at UC Berkeley’s Center for Long-Term Cybersecurity. He’s the author of the upcoming book Hidden Influences: How Algorithmic Recommenders Shape Our Lives. He previously served as a UC Berkeley tech policy fellow and a visiting AI fellow at NIST, where he contributed to standards work, co-led work on red-teaming generative AI, and advised the European Commission on recommender systems and transparency. His research continues to explore how algorithmic systems influence what people see and do online and how these systems can be audited to reduce unintended harm.
Miranda Bogen is the founding director of the AI Governance Lab at the Center for Democracy & Technology (CDT). Building on CDT’s decades of leadership fighting to advance civil rights and civil liberties in the digital age, the Lab provides public interest expertise in rapidly developing policy and technical conversations around artificial intelligence, advancing the interests of individuals whose lives and rights are impacted by AI. An AI policy expert and responsible AI practitioner, Bogen has led work at the intersection of policy, AI fairness, and governance in senior roles in industry and civil society. Previously, she guided strategy and implementation of responsible AI practices at Meta, conducted foundational research at the intersection of machine learning and civil rights at Upturn, and served as co-chair of the Fairness, Transparency, and Accountability Working Group at the Partnership on AI.
Marlynn Wei, MD, JD, is a Harvard-trained psychiatrist, psychotherapist, and author based in New York City. Integrating her background in law, ethics, and psychodynamic psychotherapy, she offers a clinical and ethical perspective on the emerging intersection of AI and mental health. Dr. Wei has testified before Congress on the need for greater transparency, accountability, and safety in AI chatbots. Her Psychology Today articles have reached nearly four million readers, and her expertise on the psychological benefits and risks of AI has been featured on CBS Mornings, PBS, and Fox Business, among other national outlets.
Moderators
Resources
Readings
- “Chatbots in Disguise,” by Ranjit Singh, Livia Garofalo, Briana Vecchione, and Emnet Tafesse, Points (2025)
- “What Happens When People Turn to Chatbots for Therapy?” by Briana Vecchione, Points (2025)
- “All the Lonely People,” by Livia Garofalo and Briana Vecchione, Points (2025)
- “AI Chatbots Need Guardrails to Protect Users’ Mental Health,” by Ranjit Singh and Livia Garofalo, Undark (2025)
- “Artificial Intelligence Is Mental: Evaluating the Role of Large-Language Models in Supporting Mental Health and Well-Being,” by Briana Vecchione and Ranjit Singh, Big Data & Society, 12(4) (2025).
- “AI Therapy Is a Surveillance Machine in a Police State,” by Adi Robertson, The Verge
References
- Testimony of Marlynn Wei, M.D., J.D. to the US House Energy and Commerce Committee, Subcommittee on Oversight and Investigations
- “The Hidden Mental Health Dangers of AI Chatbots,” Marlynn Wei, MD, JD, AI & Mental Health (Substack) (2025) | Dr. Wei’s four-domain risk framework
- “The Hidden Risk of Drift in Prolonged AI Conversations,” Marlynn Wei, MD, JD, AI & Mental Health (LinkedIn) (2026)
- “VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health,” authors include Luca Belli (2026)
- “From Symptoms to Systems: A Stakeholder-Informed Taxonomy of Generative AI Risks for Eating Disorders,” Amy Winecoff, Center for Democracy and Technology (2025)
- The Human Line Project
Credits
Production: Tunika Onnekikami
Editorial: Eryn Loeb
Design: Surbhi Chawla
Additional support provided by Data & Society’s engagement and accounting teams.


