The Limits of Human Oversight in the Age of AI Agents

Join us online on August 20, 2026 at 1 p.m. ET.

AI agents have begun to move from speculative promise into everyday deployment. They translate high-level goals into plans and carry them out across connected digital environments. Agent builders often promise that users will remain in control through approval prompts and pause buttons. Yet these safeguards work only when users can recognize a mistake and intervene before its consequences cascade. Drawing on fieldwork in a computational biology laboratory, Data & Society’s primer The Oversight Fallacy examines what effective oversight of AI Agents requires in practice.

The primer’s authors, Ranjit Singh and Samir Passi, will join Margaret Mitchell, chief ethics scientist at Hugging Face, and Madeleine Claire Elish, head of evaluations, responsible development, and innovation at Google Deepmind, to discuss why putting a human in the loop is insufficient and what builders and deployers must do to make oversight meaningful.

Accessibility

Closed captioning provided. Please email [email protected] with any other accessibility needs at least 72 hours prior to the event. Documentation, including video, transcript, and resources, will be available on our website afterwards.

Speakers

Ranjit Singh is the director of Data & Society’s AI on the Ground program, where he leads research on the social impacts of algorithmic systems, the governance of AI in practice, and emerging methods for public engagement and accountability.

Samir Passi is a social scientist, and a Data & Society affiliate, specializing in the sociotechnical and organizational challenges of building and using AI. His interdisciplinary research bridges human-centered computing and social sciences to advance effective human oversight and appropriate reliance on autonomous AI systems. Samir is currently with KForce, working as a contracted AI researcher at Microsoft.

Margaret Mitchell is a computer scientist and researcher focused on machine learning (ML) and ethics-informed AI development in tech. Her main areas of study have been natural language processing, natural language generation, assistive technology, and AI ethics. In 2023, she was recognized as one of TIME’s Most Influential People. She currently works at Hugging Face as the Chief Ethics Scientist, driving forward work on ML data processing, responsible AI development, and AI ethics.

Madeleine Clare Elish is a cultural anthropologist examining the societal impacts of AI and automation. She is Head of Evaluations, Responsible Development and Innovation, Google Deepmind. Previously she led the AI on the Ground Initiative at Data & Society, where she and her team investigated the promises and risks of integrating AI technologies.

Resources

• Passi, Samir and Ranjit Singh, The Oversight Fallacy: Why AI Agents Require More than Humans-in-the-Loop, Data & Society, July 2026, https://doi.org/10.69985/VWCK1626. 
• Passi, Samir. “Agentic AI Has a Human Oversight Problem.” SSRN Scholarly Paper No. 5529058. Social Science Research Network, September 15, 2025. https://doi.org/10.2139/ssrn.5529058.
• Elish, Madeleine Clare. “Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction.” Engaging Science, Technology, and Society 5 (March 2019): 0. NA. https://doi.org/10.17351/ests2019.260. 
• Mitchell, Margaret, Avijit Ghosh, Alexandra Sasha Luccioni, and Giada Pistilli. “Fully Autonomous AI Agents Should Not Be Developed.” arXiv:2502.02649. Preprint, arXiv, October 20, 2025. https://doi.org/10.48550/arXiv.2502.02649.
• Dhanorkar, Shipi, Samir Passi, and Mihaela Vorvoreanu. “Human Oversight of Agentic Systems in Practice: Examining the Oversight Work, Challenges, and Heuristics of Developers Using Software Agents.” Proceedings of the 2026 ACM Conference on Fairness, Accountability, and Transparency (New York, NY, USA), FAccT ’26, 2026. https://doi.org/10.1145/3805689.3812402.