Listen to this article
Estimated 4 minutes
The audio version of this article is generated by AI-based technology. Mispronunciations can occur. We are working with our partners to continually review and improve the results.
Left unchecked, artificial intelligence could kill us all — that’s the stark warning one of the godfathers of the technology, Yoshua Bengio, has been repeating for years.
On Wednesday, standing next to representatives of the Canadian and German governments, they together announced over a quarter billion dollars to help the Canadian AI researcher fight that threat.
Bengio is trying to create a new kind of AI model. The model, called Scientist AI, is being developed to be safe by design: to focus purely on objective truths and understand facts without pursuing goals of its own — like pleasing its human user. It could also act as a guardrail for other AI models.
The federal government is investing $150 million in LawZero, the Montreal-based non-profit launched by Bengio last year to build models like Scientist AI. It’s the first time that federal funding has gone to the company. Germany is providing an additional 100 million euros.
Canada and Germany will make a combined investment of up to $300 million Cdn into a non-profit group founded by Canadian computer scientist Yoshua Bengio’s group LawZero. The goal is to help develop a new form of safe AI as part of a push toward Canada’s ‘sovereign AI capability,’ according to Minister of Artificial Intelligence Evan Solomon.
“I’m doing this with the urgency that I feel in my bones, because I want my children and my grandchildren to have a future,” said Bengio at the All In conference, a major gathering of AI leaders in Montreal this week.
Concerns over AI’s risks have been mounting, reaching a fever pitch after an outgoing researcher at Anthropic — one of the AI firms considered to be at the forefront of the technology — warned there is a chance AI could wipe out humanity in the next decade.
In an interview with Bloomberg earlier this week, Prime Minister Mark Carney also called for a global, non-industry body to oversee and mitigate the technology’s risks.
Artificial Intelligence Minister Evan Soloman says the investment in LawZero is part of Canada’s larger push for responsible AI development.
“We are here today not because we are responding to yesterday’s headline, but because we are responding to the philosophic position that we have taken, which is to build responsibly and safely,” he said.
Safer by design?
In many ways, the genie is already out of the bottle, say experts: it will be hugely challenging to curb dangers such as AI hallucinating and spreading misinformation, or taking actions without explicit human permission.
“There’s no silver bullet to this problem right now. We have very powerful, even transformative, AI systems,” said Jaxson Khan, an expert in AI policy and adoption at the University of Toronto’s Munk School of Global Affairs.
American and Chinese firms operate incredibly powerful AI systems, a trend he says is likely to continue.
Anthropic CEO Dario Amodei warned that AI development needs to slow down for safety measures to catch up. U.S. President Donald Trump dismissed the risk of artificial intelligence when questioned by reporters.
“Canada’s at this moment where we’re feeling like perhaps, in the context of AI, we don’t have a lot of control,” he said.
But teaming up with Germany to invest in LawZero is still meaningful, Khan says.
“You have to be able to develop alternatives that ideally are safer by design,” he said.
“I’m curious to see if that will work, but I do think it’s a worthwhile investment from the government and a good partnership with Germany.”
Domestic capacity and partnerships
Another important piece of the puzzle, experts say, is partnering with countries that have similar goals and values.
The announcement came hours after two AI companies, Canada’s Cohere and Germany’s Aleph Alpha, signed a merger agreement.
“This is arguably capacity building,” says Vass Bednar, the managing director of the Canadian Shield Institute think-tank, of the merger.
“We do have capacity to catch up and build alternatives. We also have to believe that,” she said.
“We can’t … fall victim to the inevitability of the technology or its harms.”
Taking a multipronged approach could make a difference, says Khan.
Bengio says giving up isn’t an option.
“It won’t be easy to achieve our goals,” he said. “But if we can change the world even by nudging it a little bit towards a better world for humans, we have to try.”
