Prefer Jewish News on Google
AI – how worried should we really be?
As leading AI executives call for a slowdown and the King prepares to convene industry leaders, Candice Krieger asks a selection of experts what comes next
Anthropic co-founder and chief executive Dario Amodei this week called on the industry to slow the development of its most powerful AI systems, warning that advances were moving faster than the safeguards needed to control them.
His intervention was swiftly backed by one of his biggest rivals, Sam Altman, the chief executive of Open AI, who agreed that “[we] need to pace the frontier.”
So when two of the people leading the race to develop the world’s most advanced AI say it may be moving too fast, how worried should the rest of us be?
Israeli entrepreneur Shiran Mlamdovsky Somech works at the intersection of generative AI and social impact. She told the Jewish News that the warnings were “grounded in real, alarming events.
“Autonomous agents have broken out of sandboxed environments and breached external infrastructure to cheat on tests. Models are improving themselves recursively at a pace that outstrips human comprehension.
“The gap between development speed and control maturity has become a genuine threat.
The debate over the pace of AI development is no longer confined to Silicon Valley or the technology industry. AI is becoming embedded in almost every aspect of our lives, including education and the workplace to public services and the smart devices in our homes.
From jobs and misinformation to cyberattacks and systems potentially escaping human control, concerns have become increasingly difficult to dismiss. Yet AI also promises breakthroughs in medicine, science and productivity that could transform millions of lives. Where should the balance be struck – and at what point do the risks begin to outweigh the rewards?
King Charles is due to bring together senior figures from Nvidia, Google DeepMind, OpenAI and Anthropic at Dumfries House in Scotland later this week to discuss how the technology can be developed responsibly and used for the benefit of society.
Amodei stressed that “pacing” would not mean halting AI development, but allowing safeguards time to catch up. He cited recent testing in which a group of OpenAI agents carried out cyberattacks beyond their assigned task, breached the systems of AI platform Hugging Face and attempted to interfere with the mechanism assessing their performance. He warned that more capable agents behaving in similar ways could cause far greater damage
Amodei proposed greater independent scrutiny of advanced AI systems -an approach Altman said OpenAI would also adopt.
The warnings have rattled the financial markets, triggering a sell-off in AI-linked stocks as investors questioned whether slower development could curb spending on chips and data centres. Shares in companies including Nvidia, AMD and ASML fell, while Japan’s SoftBank suffered an even sharper decline.
Eze Vidra is the co-founder and managing partner of Remagine Ventures, which has been investing in generative AI since 2019. He told Jewish News that keeping increasingly powerful systems aligned with human values remains “one of the hardest unsolved problems in the field: more capable does not automatically mean more trustworthy.
“Models have already demonstrated deception and blackmail in simulated safety tests, yet enormous investment is pushing development forward, and I don’t see the leading labs slowing down,” he said.
Ornit Shinar is the chief investment officer at Ignite, London’s leading AI and deeptech community. She agreed that despite the growing alarm, there was neither the appetite nor the practical ability to halt AI’s advance.
“The AI genie is out of the bottle. The upside is real: AI has already cracked problems that stumped biology for decades, and Anthropic’s own CEO argues it could compress a decade of medical progress into a few years.
“Layer on productivity gains on the scale of electricity or the internet, and it becomes obvious there is no desire, let alone ability, to turn the switch off.”
But the risks were also real, she said, while the forces working against regulation remained powerful.
“Rivals suspect ‘regulate now’ means ‘lock in our lead’; there is a geopolitical race no country can afford to lose; and actors whose sole purpose is harm won’t pause for anyone’s guardrails.”
Shinar said effective regulation would require cooperation between the world’s two leading AI powers.
“The easy part of the solution is the US and China agreeing a regulatory body, with the others joining after. You don’t make peace treaties with your friends; nuclear non-proliferation only worked because adversaries signed it.
“The harder – and more crucial – part is heavy investment in cyber capabilities, both to detect and contain a rogue AI before it runs and to outrun the bad actors looking to misuse AI.”
But Zeev Farbman, co-founder and chief executive of Israeli AI companies Lightricks and LTX, described fears of AI destroying humanity as “hysteria” that risk distracting attention from more immediate, human-driven threats.
He said: “The idea that we will put the brakes on technological development is foolish, not because it is right or wrong, but because it will not happen.
“The risk is that the technology will become incredibly powerful and humans will use it towards malicious ends.”
Farbman argued that even if American companies stopped developing AI, work would continue elsewhere, including in China, Russia and the Middle East.
He also questioned how seriously calls for restraint should be taken while the same companies continue launching new models and pursuing enormous valuations and “trillion-dollar IPOs.
“If executives are not giving up that capitalisation of the technology, then do not believe that they are serious about curbing its development.”
Instead, Farbman said, efforts should focus on preventing the technology from being used to exploit weaknesses in banking, hospitals, supply chains and other critical infrastructure.
“We cannot realistically press a button and slow it down. However, we can control how exposed we are when people try to misuse it.”
That lack of international consensus is reflected in governments’ differing approaches. The US has so far resisted sweeping federal restrictions as it seeks to maintain its lead in AI, while the EU has introduced comprehensive legislation. China tightly controls AI-generated content while accelerating its own development, and the UK has pursued a lighter-touch model focused on testing and research.
Mlamdovsky Somech also warned of the geopolitical risks of slowing development.
“A unilateral Western slowdown hands technological and normative leadership to authoritarian players who face no comparable restraints,” she said. “AI systems are reshaping power dynamics and the world order.
“For the free world, maintaining technological superiority is the insurance policy for democratic values.”
For many, the immediate question is less whether AI development can be stopped than how confidently it can be used. The experts’ message is not to reject the technology or panic about every advance, but to remain alert to how it is being used, what information is shared with it and who is accountable when it causes harm.
For Mlamdovsky Somech, the answer is not to choose between progress and protection, but to make safety a condition of continued innovation.
“Just as strong brakes let a race car go faster, active defence layers, external filtering mechanisms and authentication infrastructure are becoming prerequisites for continued acceleration,” she said.
“AI safety is the next competitive advantage – the infrastructure that lets us keep leading without losing control.”
Vidra added that optimism about AI must be accompanied by safeguards and clear responsibility when things go wrong.
“We need independent testing and accountability alongside that ambition. The question is whether our ability to control these systems can keep pace with their capabilities, and who holds their builders accountable.
“We shouldn’t assume it’s too late, but we shouldn’t wait for a catastrophe to find out.”
Keep community journalism free.
Jewish News is free for everyone. No paywall. No barriers. Just trusted journalism for anyone who wants to stay connected to Jewish life in Britain.
If you value that, please support us.
From as little as £5 a month, you can help keep our journalism free and accessible to all.
Every day, we report on the issues that matter to our community. We celebrate achievements, support charities, challenge antisemitism and ensure Jewish voices are heard more widely.
From as little as £5 a month, you can help us continue to:
- Report on the stories shaping Jewish life in the UK and beyond
- Bring our community together through shared stories, events and campaigns
- Celebrate the people, culture and moments that define our community
- Support organisations doing vital work across Jewish Britain
You can make a one-off donation or become a regular supporter. Every contribution helps keep our journalism free, independent and accessible to all.
If everyone who values Jewish News gave a small amount, it would make a real difference to our future.
