Artificial intelligence whistleblowers painted a dismal picture about the direction the technology is going at a City Council hearing Monday, warning local lawmakers to put in place regulations to slow its growth — even as the companies insist they’re operating with caution.
The rare Committee of the Whole meeting came as public fears around AI have spiked and the Trump administration has taken a lax stance on the rapidly developing technology. At the hearing, industry leaders warned of potential dangers like bioengineered plagues, or even an end to humanity.
Former Anthropic researcher Jacob Coxon testified that AI could “could make nearly everyone’s life better” with advancements in science and medicine.
“But on the current path, I think it is more likely than not that humanity loses control to these AIs, and it could end in human extinction,” Coxon told the Council, adding that companies driving this process like OpenAI and Anthropic “need to take safety much more seriously.”
Executives from OpenAI, Anthropic, Google and Meta also testified during the marathon hearing, facing dubious lines of questioning from dozens of councilmembers.
Their testimony on Monday was the first such appearance before any legislative body. But local lawmakers, even for the largest city in the country, have only limited tools with which to regulate the internationally operating companies. Menin said she hopes the Council’s work can serve as an example for lawmakers in D.C.
“The federal government has opted not to regulate the AI industry. They should,” Council Speaker Julie Menin said at a press conference ahead of the hearing. “So we believe we have a package of ideas that are really good, and that we hope can serve as a model for the federal government or for other states.”
Menin appeared skeptical of the company representative’s testimony at many parts of the hearing after she started the hearing by asking the executives to quantify the risk of the worst-case scenario outlined by the whistleblowers.
“I don’t know. I also don’t think it matters,” Morgan Dwyer, head of policy at OpenAI, replied, adding that no chance is “remotely acceptable.”
Menin responded that she was “incredulous” at this answer: “I’m sorry, but to say you don’t know and it doesn’t matter is flippant at best.”
The Open AI rep claimed to have supported the RAISE act — a state law to regulate AI models that was passed last year — a claim others refuted, arguing the executives had perjured themselves under oath.
“I was there — they actively pushed to water down the bill’s safety requirements,” State Sen. Andrew Gournades said. “It’s another example of why we can’t trust these companies to regulate themselves.”
Ex-Google DeepMind researcher Alex Turner told the Council he fears the tech may soon evolve to the point where a superintelligent AI swarm could “wrest control of human civilization,” placing the odds of a complete AI takeout at around one in three.
And former OpenAI researcher Daniel Kokotajlo warned that AI’s development could turn into a feedback loop where the technology autonomously designs and codes the next versions of its system, leaving humans to play a cursory role akin to a “board of directors.”
“This is a recipe for disaster,” Kokotajlo said, arguing that AI development should be slowed down dramatically while transparency measures are increased.
The companies, for their part, emphasized that they value safety and security as they develop new products.
“We have been, I think, sometimes accused of being overly cautious,” Logan Graham, the head of Anthropic’s Frontier Red Team, argued, adding that the company had withheld its most powerful model to date this year “in order to just have more time to even think about… what to do about these capabilities.”
The Council is currently weighing 10 pieces of legislation aimed at regulating the AI industry. Among those is a bill requiring AI companies operating in New York City to have a human “kill switch” function, which would need to be verified by a third party. Violations would carry $25,000 fines, per the bill.
Other bills being discussed including one that would pay AI whistleblowers a portion of the fines recovered from companies, allow citydwellers to sue if a company had insufficient safeguards that led to malicious behaviors, and require companies contracting with the city to report AI safety incidents within 24 hours.
Meanwhile, President Trump has largely brushed aside warning about the dangers of AI, instead putting an emphasis on winning a technology arms race against China. Last week, he met with executives from major AI companies at the White House, where he said they signed a voluntary accord to regulate.
Mayor Mamdani, asked about the hearing on Monday, said he was supportive of the Council’s efforts to regulate AI but emphasized that more oversight must come from the federal government.