Turing Prize winner urges governments to require tech groups to cover catastrophic outcomes and fund safety research
Artificial intelligence companies should be compelled by law to have liability insurance to cover existential risks, according to one of the key pioneers of the revolutionary technology.
Speaking at the FT Future of AI Summit in London on Thursday, Yoshua Bengio warned that AI companies had few financial incentives to build safer AI systems and that this required governments to make an urgent market intervention.
The Canadian computer scientist and Turing Prize winner encouraged politicians to mandate liability insurance for AI companies, as is done with developers of nuclear power plants. He also called for an increase in research and development investment for “guardrails” that protect people from catastrophic events, such as AI being used to build new bioweapons.
Asked what made him think companies cannot be relied upon to build such guardrails, Bengio said: “I don’t know, but I don’t want to bet the future of my children on it.”
The computer scientist is among the so-called godparents of AI — whose research was relied upon to build generative AI systems created by OpenAI, Anthropic and Google — who have grown increasingly concerned about the risks associated with the technology.
Bengio’s demands for government intervention come as OpenAI and Anthropic consider using investor funds to settle potential claims from multibillion-dollar lawsuits, as insurers balk at providing coverage for the risks associated with AI.
People close to the situation say the reason insurance companies are reticent to provide comprehensive cover for AI companies comes from the unprecedented scale of potential claims faced by relatively young tech companies.
Fei-Fei Li, who, alongside Bengio and other AI pioneers was awarded the 2025 Queen Elizabeth Prize for Engineering this week, said: “It’s not seven companies’ responsibility and it’s not only a few individuals who know the technology. It’s all of our responsibility” to develop AI with human dignity at its core.
Asked what she thought of the reputation for Big Tech to “move fast and break things” — an early motto at Meta’s social network Facebook — Li said: “That’s just one company,” adding there were academics and other key figures seeking to rein in the excesses of Big Tech groups.
“I’m going to defend the AI sector a little bit,” she said. “I think the AI sector is very big and the AI ecosystem is not just a few companies.”
But Bengio said AI systems had already been shown to be powerful tools that can create military weapons and have shown “deceitful” traits that mean they could turn against human users.
He was concerned that instead of focusing on safety, tech companies are “all competing with each other in this very tight race where they want their AIs to perform better on benchmarks, [to] get more engagement from users, because they want to make sure they maintain or improve their market share”.
Helen King, vice-president for responsibility at Google’s AI arm DeepMind, said companies were paying attention to so-called frontier safety risks. She also warned that “a really great coding agent also increases the risk of it being really great at cyber attacks”.
The debate over AI safety came a day after Nvidia chief Jensen Huang told the FT: “China is going to win the AI race.”
The head of the AI chipmaker and the world’s most valuable listed company said that the west, including the US and UK, was being held back by “cynicism” as Beijing loosens regulations and cuts energy costs for data centres.
Responding to Huang’s comments, Bengio said the US-China AI race was “a very unhealthy direction because it’s an incentive for both players to cut corners on public good and safety”.
Source: https://www.ft.com/content/181f6706-1b23-4691-96bd-db3ae2e3e6bf
