NYC Council AI Hearing Opens: Whistleblower Testifies Labs Are "Extremely Reckless," Humanity "More Likely" to Lose Control

What Happened

The hearing convened as a Committee of the Whole, with all 51 council members present, to examine AI risks and legislative proposals. Representatives of Anthropic, OpenAI, Google and Meta testified under oath before a US legislative body for the first time — only Meta came voluntarily; Anthropic, OpenAI and Google confirmed only after the Council threatened subpoenas. SpaceXAI (Musk's company), which did not respond to the Council's invitation at all, was served a subpoena that same day.

Two groups sat at the witness table. The whistleblowers: former Anthropic researcher Jacob Coxon (27, who did pretraining research at OpenAI and Anthropic for three years and resigned publicly on September 8, accusing both companies of "gambling with our lives"), plus former Google DeepMind researcher Alex Turner and former OpenAI researcher Daniel Kokotajlo, who runs the AI Futures Project. The labs' side: Logan Graham, head of Anthropic's Frontier Red Team; Morgan Dwyer, OpenAI's head of policy development and operations; Shane Cahill, Meta's AI policy director; and Alice Friend, Google's AI policy executive.

Key Facts

1. Coxon's charge: "From my experience, the companies are being extremely reckless given the stakes," Coxon said. "The companies run on a startup mindset: move fast, break things, fix them later. That works for a photo sharing app. It does not work for building the most powerful technology ever built." He added that the majority of the code is now written by AI, and "people do not check it that carefully anymore."
2. "More likely" to lose control: "On the current path, I think it is more likely than not that humanity loses control to these AIs and it could end in human extinction," Coxon told the Council. He also said: "We don't understand its drives or why it does the things it does," and that AI is approaching the point where systems become capable of improving themselves.
3. One-in-three takeover odds: Former Google DeepMind researcher Alex Turner testified: "A superintelligent swarm could wrest control of human civilization... I myself would guess AI takeover chances at roughly one in three."
4. The labs' rebuttal: The four companies' representatives testified in the same room that their systems are already improving people's daily lives, and stressed that they are "laser-focused on safety."
5. The bills on the table: The Council is weighing a package introduced on September 25: mandatory third-party validation for AI systems used in the city; nation-leading whistleblower incentives; and a private right of action for individuals harmed by AI systems. Reported details include a $25,000 penalty for each missing or falsified validation, and 24-hour reporting of covered safety incidents involving city contracts. The bills have not passed yet.

Context

The hearing had been building for weeks: Coxon's public resignation and extinction warning on September 8, the Council's formal announcement on September 28. At the federal level, Trump has dismissed AI fears as a "hoax" and late last month signed a voluntary safety agreement that largely lets the companies self-police (previously reported here); Congress has been mostly inactive. New York City tabled its bill package on September 25, and the hearing was convened under Council Speaker Julie Menin.