Data & Society > our work > newspaper article > Breaking the Black Box: When Machines Learn by Experimenting on Us

ProPublica | 10.12.16

Breaking the Black Box: When Machines Learn by Experimenting on Us

Julia Angwin, Terry Parris Jr., Surya Mattu, Seongtaek Lim

D&S affiliate Surya Mattu, with Julia Angwin, Terry Parris Jr., and Seongtaek Lim, continue the Black Box series.

Depending on what data they are trained on, machines can “learn” to be biased. That’s what happened in the fall of 2012, when Google’s machines “learned” in the run-up to the presidential election that people who searched for President Obama wanted more Obama news in subsequent searches, but people who searched for Republican nominee Mitt Romney did not. Google said the bias in its search results was an inadvertent result of machine learning.

Sometimes machines build their predictions by conducting experiments on us, through what is known as A/B testing. This is when a website will randomly show different headlines or different photos to different people. The website can then track which option is more popular, by counting how many users click on the different choices.

Julia Angwin

Surya Mattu

Terry Parris Jr.

Seongtaek Lim


Subscribe to the Data & Society newsletter

Support us

Data & Society Research Institute 36 West 20th Street, 11th Floor
New York, NY 10011, Tel: 646.832.2038

Reporters and media:
[email protected]

General inquiries:
[email protected]

Unless otherwise noted this site and its contents are licensed under a Creative Commons Attribution 3.0 Unported license.