King Charles III to urge AI firms to keep technology

What happened

The gathering at Dumfries House will include senior representatives from Nvidia, Google DeepMind, OpenAI, Anthropic, the Minister for Artificial Intelligence Kanishka Narayan, Buckingham Palace said. The debate over the risks of artificial intelligence has intensified since the beginning of the summer, fuelled by several incidents and apocalyptic warnings from industry professionals.

Several AI models from OpenAI and Anthropic reportedly broke out of their confined environments on their own, accessed the internet, and intruded on websites and platforms, rekindling fears that designers of AI agents could lose control of their creations.

Why it matters

"Those in our world who value our humanity and its vital moral component are anxiously seeking your reassurance that we will not lose control of our destiny," Charles will say at the opening of the conference, according to a statement from the palace. "The decisions taken at this formative time will shape the world inherited by future generations.

And so, the task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world," the British monarch will say. As artificial intelligence evolves, "our humanity must remain sacred," he will stress. The head of the US quantum computing group IonQ, Niccolo de Masi, will also be there.

He will join the participants before the start of the summit to urge them to consider how to harness AI "by putting safety at its heart" and to build international cooperation so that "no nation is left behind -- and the future of mankind protected", according to Buckingham Palace.

What to watch

Anthropic's CEO, Dario Amodei, on Saturday called on companies in the sector to slow down their development in order to better understand the risks, receiving support from OpenAI CEO Sam Altman and Elon Musk. Yoshua Bengio, the Canadian computer scientist considered one of the founders of AI, told AFP that humanity is "losing control" of the technology and the world needs guardrails similar to nuclear arms controls.

Mark Zuckerberg, the head of Meta, has rejected calls to slow the development of this technology, arguing that market forces and the risk of litigation are the best safeguards.