The Rise of AI-Supported Healthcare: Safeguarding Patients and Healthcare Workers
Healthcare has always been shaped by power: who is listened to and insured, who bears the risk, and who profits from these systems. With the rise of digital platforms and AI, we are seeing these dynamics accelerate without corresponding protections for healthcare workers and patients. Over the past several years, Data & Society has examined what happens when technology compounds existing harms in care settings, who gets left out, and what accountability could actually look like.
We started this work by listening to healthcare professionals working alongside these systems. Our researchers spoke with therapists navigating the rise of teletherapy platforms and found that the changing conditions of therapeutic labor are gradually redefining therapeutic work. We have also chronicled the integration of AI tools into clinical settings, such as the integration of Sepsis Watch into the Duke Health system, and repeatedly saw the same result: these systems were eroding the professional norms that make good care possible in the first place.
We then turned to how patients, both those in clinical settings and those seeking help outside them, are being impacted by new technologies. Our research into data infrastructures and the Black birthing experience uncovered how technology is reshaping the delivery of maternity care and how one of the most acutely harmed groups in American healthcare is bearing the cost.
Most recently, we’ve been focusing on one of the fastest-moving and least-regulated arenas in digital health: how AI chatbots are being used as a tool for mental health support. Data & Society researchers have spent over a year talking to chatbot users, therapists, clinicians, designers, developers, and policymakers to understand how people turn to these tools for emotional support and mental health care. What they found, as researcher Dr. Briana Vecchione writes, is that people aren’t confused about whether chatbots are human, and they are not unaware of the risks to their data. What draws them in are affordances that map onto prior failures in the care landscape, including availability, low cost, perceived privacy, and freedom from judgment. The same features that make these systems feel trustworthy and accessible are those that produce harm in the absence of clinical accountability. As researchers Dr. Ranjit Singh and Dr. Livia Garofalo wrote in Undark:
“The issue is not just that the bots talk; it’s that the system is designed to keep you talking. This form of predatory companionship emerges in subtle ways. Unlike a mental health professional, chatbots might ignore, or even indulge, risk signals such as suicidal ideation and delusional thinking, or offer soothing platitudes when urgent intervention is required.”
That dynamic is especially dangerous for people in crisis, for whom the response of a clinically unqualified chatbot may be all they have to turn to. Data & Society took that case directly to policymakers through a March 2026 policy brief recommending design constraints and accountability mechanisms at the state level; in a formal comment submitted to the FDA, we argued that if a chatbot looks and acts like therapy, a disclosure requirement notifying users that they are interacting with AI is not enough regulatory protection. Later in 2026, we are releasing a report that brings all this work together, providing the concrete empirical understanding required to make more equitable decisions about how this care is designed, used, evaluated, and regulated.
Across all of this work, the throughline is the same: technology is being deployed in care contexts faster than accountability structures can keep up. Data & Society works to help close that gap by centering the experiences of the workers, patients, and communities who feel the consequences first.