Just a few years ago, Soizic Pénicaud was working for the French government to document which agencies were deploying algorithms and for what purposes, and creating resources to help residents understand how their government was using AI. Now she’s calling attention to the technology’s potential abuses from the outside.
Pénicaud, 31, grew frustrated that some algorithms—including ones she believed posed the greatest threats, like those wielded by police and welfare administrators—were exempt from the public evaluation process. “I felt like we weren’t solving the problem of harm, even if we were working on transparency,” she says.
She left government in 2021 to become an independent researcher and AI policy specialist. In 2023, she partnered with the nonprofit newsroom Lighthouse Reports to investigate algorithms run by France’s social security agency to rate welfare recipients on how likely they were to commit fraud. By obtaining and examining the source code of three related models, Pénicaud and her collaborators showed that the system unfairly targeted single parents and people with disabilities. Their findings sparked a lawsuit against the agency by 15 NGOs. (MIT Technology Review recently collaborated with Lighthouse on an unrelated story about a welfare algorithm in Amsterdam.)
Her latest project is to build a public repository describing algorithms deployed by the French government. It launched last November, detailing 72 programs, including a system that flags restaurants for health inspections based on online reviews, another that hospitals use to prioritize patients for heart transplants, and several deployed by welfare and law enforcement agencies. She knows of at least 50 more.
Her efforts now extend beyond France, too—working with a nonprofit coalition based in Spain, and an informal group of representatives from 15 governments that meets to discuss AI governance.
Pénicaud does see a role for AI in government—to more efficiently allocate resources, like energy, or to plan urban improvements—but she believes it’s the responsibility of public officials to prove that such systems work as intended. And that they don’t harm people in the process.
