Questions mount over what an AI 'slowdown' would look like

Image source, ReutersImage caption, Anthropic CEO Dario Amodei has called for the pace of development to be slowed down

Tech and AI EditorPublished5 hours agoThere has been no shortage of warnings from AI's top bosses about existential threats over the years.

I attended the world's first AI safety summit in Bletchley Park in November 2023 where the focus was on the most serious threats imaginable, posed by the most powerful AI models.

At the time, many thought this was in the realm of sci-fi and the real threats were comparatively far more mundane: job losses, cheating in exams. Three years later, many of those same voices don't seem to be so sure.

On Saturday head of AI company Anthropic, Dario Amodei, urged the pace of development to be slowed down - the bosses of two rival AI firms, Sam Altman of OpenAI and Elon Musk said they agreed with him.

Meanwhile AI researcher Jacob Coxon - who quit Anthropic - told the BBC on Sunday staff who were developing the systems were "genuinely frightened" for the future of humanity.

They all add up to a chorus of serious voices urging a slowdown, but although it might sound like a quick fix, it is far from an easy solution.

AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

Published9 hours agoAmerica is widely understood to be terrified of losing the race to build the most powerful AI to China.

On Sunday President Trump made his own views clear, claiming the US was ahead of China in the AI race and declaring: "Whoever wins AI, wins."

China isn't renowned for wanting to come second. Everyone I speak to says that even if their company pauses, others will not and they will just get left behind. It reminds me of the height of the campaign for nuclear disarmament. The problem then too was that nobody wanted to go first.

There is also the question of how on earth a slowdown would be enforced. Who would police it? Would we be relying on the AI companies to be transparent about what they are doing, and more importantly what they are choosing not to do? That would require an epic level of trust, which arguably the tech sector has never earned.

Although Amodei in his post proposed a three-point plan that included independent monitoring of AI models as they are developed, industry-wide regulation and global regulation, some observers question how this would work in practice.

"Nobody has given a substantive explanation of what 'slowdown' means," said Ed Zitron, CEO of EZ Primary Research.

"Putting a plant from METR, where one of the guys who quit Anthropic went, in every AI company? Democratic and global 'co-ordination'? Doesn't mean anything to me."

Zitron said stopping training AI models would leave them vulnerable to Chinese rivals.

"Their margins might improve but their products will be captured in amber and distilled by Chinese labs. They may have to change pricing. I truly don't know how it all works.

"This could burst the bubble. But right now we are very thin on what a 'slowdown' means."

To play this video you need to enable JavaScript in your browser.This video can not be played

'There is a possibility of human extinction,' former Anthropic employee says about AI

In the UK, we have heard about plans to roll out the tech more widely within the NHS to improve patient care, and how AI gave the UK economy a much-needed boost over the summer. We are all being encouraged to use it more and more at work, in education and in our personal lives.

The industry is a key part of the UK's strategy to drive economic growth. "There is no plan B" a former government adviser told me, so could a slowdown wreck future prospects here in Britain?

The AI industry is burning through enormous amounts of money and natural resources and so far, is not creating nearly as much revenue. There are multiple reports suggesting a number of firms adopting the technology are disappointed by it.

Economists widely speculate that even the current giants are not all likely to survive and that some kind of "levelling" is coming – also known as a bursting bubble. But those firms which do make it could end up becoming the most powerful mega-corporations the world has ever seen, and that comes with its own issues.

OpenAI choosing to halt its march to the stock market could be seen as an epiphany-like moment of taking responsibility for public safety – but it could also be the move of a company which has realised it might not get the lucrative payout it needs while its entire product is perceived to be lethal.

"One of probably the biggest challenges that we have right now is that no one can predict with certainty how this technology is going to evolve," says Alexander Voica from the UK AI company Synthesia.

"We know that these systems are getting more powerful, but we don't know where and how they're going to be used, and we haven't figured out essentially a way of taking full advantage of their potential. My only concern in rushing to regulate now, where there's still a lot of open questions, is that it could actually backfire."

We also can't deny the fact that underpinning this entire tech revolution is an ocean of investor cash.

"I'm not worried about the existential risks of AI, I'm worried about the corporate greed of the companies that are creating it," says Sasha Luccioni, the founder of Sustainable AI.

Leading computer scientist Professor Dame Wendy Hall, who has advised the UN about AI, says the current crisis is all about the companies not behaving responsibly enough.

She likens the current situation to a farmer having a bull in a field which escapes and causes destruction – and the farmer blames the bull.

"Of course it's not the bull's fault – it's the farmer," she says. Clearly, the fences weren't robust enough, and that is exactly what she says we are seeing with AI guardrails right now.

But if those guardrails become too restrictive, could this spell the end of AI?

Some believe there is a more secretive, politically-motivated push for a rules-based clampdown in order to "regulate AI into oblivion" as Parker Thayer, an investigative researcher at the Conservative-leaning Capital Research Centre think-tank, put it on X this week.

His post was viewed nearly eight million times. It is an extreme and unproven view but it shows that not everybody is on board with the idea of regulation saving the day.

Whatever the reality, the storm currently engulfing the AI industry could already have caused reputational damage to the firms currently in pole position forever.

As Professor Hall puts it: "Would you invest in a company that says it's going to bring about human extinction?"

Additional reporting by Tom McArthur

Anthropic boss Dario Amodei calls for AI development to slow down

Dramatic insider warnings over AI fall flat with some in Silicon Valley

Trump downplays AI risks after dire expert warnings and calls to slow development down

The video playlistWatch our pick of standout clips from across the BBC

PreviousNext1:06Why is Trump lifting Irish whiskey tariffs? 00:01:06, play videoWhy is Trump lifting Irish whiskey tariffs?0:48Tom Felton on Harry Potter Remake. 00:00:48, play videoTom Felton on Harry Potter Remake1:00Burnham first serving PM to take part in Great North Run. 00:01:00, play videoBurnham first serving PM to take part in Great North Run1:25Teri Hatcher on playing 'iconic' Miranda Priestly. 00:01:25, play videoTeri Hatcher on playing 'iconic' Miranda Priestly1:36Does this AI comic make you laugh? 00:01:36, play videoDoes this AI comic make you laugh?0:49Are the UK's spiders getting bigger? 00:00:49, play videoAre the UK's spiders getting bigger?0:57'I was a commodity': Charlotte Church on the music industry. 00:00:57, play video'I was a commodity': Charlotte Church on the music industry0:23Things heat up in the Celebrity Masterchef kitchen. 00:00:23, play videoThings heat up in the Celebrity Masterchef kitchen0:57What just happened with Trump's united Ireland call. 00:00:57, play videoWhat just happened with Trump's united Ireland call0:50Wembley Stadium's first ever Punjabi artist headliner. 00:00:50, play videoWembley Stadium's first ever Punjabi artist headliner0:59When a Dragon gives you a helping hand. 00:00:59, play videoWhen a Dragon gives you a helping hand1:02Why should young people care about pensions? 00:01:02, play videoWhy should young people care about pensions?1:30My attacker 'doesn't get to be the storyteller' 00:01:30, play videoMy attacker 'doesn't get to be the storyteller'1:07The 3D scans giving blind parents a new way to meet their baby. 00:01:07, play videoThe 3D scans giving blind parents a new way to meet their baby1:11Are women better at endurance running than men? 00:01:11, play videoAre women better at endurance running than men?1:20Why is Donald Trump visiting the Republic of Ireland? 00:01:20, play videoWhy is Donald Trump visiting the Republic of Ireland?0:54'This ceremony is essential' says brother of 9/11 victim. 00:00:54, play video'This ceremony is essential' says brother of 9/11 victim1:31Inside Gander: The town that took in thousands on 9/11. 00:01:31, play videoInside Gander: The town that took in thousands on 9/110:57'Extraordinary' - MPs reject assisted dying bill. 00:00:57, play video'Extraordinary' - MPs reject assisted dying bill1:17Céline Dion duets with fan ahead of Paris comeback concert. 00:01:17, play videoCéline Dion duets with fan ahead of Paris comeback concertTop storiesRussia hits Ukrainian train shortly after Boris Johnson and top European officials leave station

'Culture shift' needed in how UK does business, PM urges

Trump downplays warnings of AI risks, citing rivalry with China

I had 11 years of chemotherapy for a cancer I didn't have

Why I'm doing a September reset and how you can create a new routine

The Papers: 'Compulsory lentils' and 'Boris flees Vlad drone blitz'

How to protect your laptop, phone and bike from thieves at uni

Buckingham Palace works uncover trove of everyday items dating back to 1800s

Emmy Awards: How to watch, who is nominated and why Heated Rivalry is missing

Carney gambles on the world's biggest investors betting on Canada

The Himalayas are melting faster - and India's economy is 'at risk'

US Politics Unspun: Cut through the noise with Anthony Zurcher's newsletter

How a woman's sudden death triggered a six-month manhunt

Abercrombie & Fitch's former CEO Mike Jeffries to stand trial

Dvorak’s epic masterpiece at the Proms

Greg Jenner and guests examine the tumultuous events of 1066

Russia hits Ukrainian train shortly after Boris Johnson and top European officials leave station

Meghan shares glimpse of family life after Sussexes return to UK

'Culture shift' needed in how UK does business, PM urges

Deep-fried food banned under new school dinner rules in England

Gloomy forecast for tenants as rent rises set to speed up

I had 11 years of chemotherapy for a cancer I didn't have

Why I'm doing a September reset and how you can create a new routine

Trump downplays warnings of AI risks, citing rivalry with China

Will record donations to Reform go down as a moment of political change?

Trump says he will remove all Irish whiskey tariffs as he ends two-day visit