Dan Kitwood/Getty Images
- Google DeepMind CEO Demis Hassabis says scaling laws are vital to the tech’s progress.
- Scaling requires feeding AI models ever more data and more compute.
- Some other AI leaders, however, believe the industry needs to find another way.
There’s a debate rippling through Silicon Valley: How far can scaling laws take the technology?
Google DeepMind CEO Demis Hassabis, whose company just released Gemini 3 to widespread acclaim, has made it clear where he stands on the issue.
“The scaling of the current systems, we must push that to the maximum, because at the minimum, it will be a key component of the final AGI system,” he said at the Axios’ AI+ Summit in San Francisco last week. “It could be the entirety of the AGI system.”
AGI, or artificial general intelligence, is a still theoretical version of AI that reasons as well as humans. It’s the goal all the leading AI companies are competing to reach, fueling huge amounts of spending on infrastructure and talent.
AI scaling laws suggest that the more data and compute an AI model is given, the smarter it will get.
Hassabis said that scaling alone will likely get the industry to AGI, but that he suspects there will need to be”one or two” other breakthroughs as well.
The problem with scaling alone is that there is a limit to publicly available data, and adding compute means building data centers, which is expensive and taxing on the environment.
Some AI watchers are also concerned that the AI companies behind the leading large-language models are beginning to show diminishing returns on their massive investments in scaling.
Researchers like Yann LeCun, the chief AI scientist at Meta who recently announced he was leaving to run his own startup, believe the industry needs to consider another way.
“Most interesting problems scale extremely badly,” he said at the National University of Singapore in April. “You cannot just assume that more data and more compute means smarter AI.”
LeCun is leaving Meta to work on building world models, an alternative to large-language models that rely on collecting spatial data rather than language-based data.
“The goal of the startup is to bring about the next big revolution in AI: systems that understand the physical world, have persistent memory, can reason, and can plan complex action sequences,” he wrote on LinkedIn in November.
Â
