Have we reached peak AI?
#111 - Are We at the End of the AI Scaling Revolution?
Ciao, I am Giovanni and I scribble thoughts weekly on all kinds of stuff I find interesting. If this post made you smile or think, and you’re not already on the list, hit the subscribe button below. It’s free, and no, I won’t spam your inbox!
Hello friends, I hope you’re doing well!
This week I have done a bit of traveling for work and a bit of running, which proved to be the perfect occasion to listen to a great 5 hours long podcast with the Anthropic team. I am a big fan of this long-conversation format that Lex Fridman has established, and this was a really good one.
This episode has a lot of great content, but the one reason it got me hooked is that in the opening part it tackles something that has been widely reported and commented lately in AI news, and that I wanted to understand better: scaling laws are not working anymore as they used to.
The last few years advancements of AI, that felt incredibly fast, were mainly based on the fact that making models bigger, training them on more data, and throwing more compute at the problem, made them exponentially better. This is what brought us in a few months from rudimentary chatbots to systems like GPT-4, capable of passing professional-level exams and generating coherent reasoning.
And now a lot of people, including Dario Amodei (CEO of Anthropic), argue that these laws might be hitting a wall.
This matters because many experts are predicting AGI—artificial general intelligence—within the next few years. If we’re on the brink of AGI, understanding the limits of scaling and the alternatives we need to pursue is not just a technical question—it’s a crucial point for AI future.
What Are Scaling Laws?
Scaling laws have been the silent engine of AI’s extraordinary progress. They describe a predictable relationship between three key variables: model size, dataset size, and compute. Increase these variables in the right proportions, and performance improves— often in leaps rather than steps.
Amodei described scaling laws with a compelling analogy: they’re like balancing a chemical reaction. You can’t just add more of one reagent (say, compute) without the right amounts of the others (data and training time). But when all three align, the results are extraordinary. This framework is what allowed models like OpenAI’s GPT series to grow from simple language processors to systems capable of complex reasoning, code generation, and multimodal tasks.
Critically, scaling laws were very reliable. Researchers discovered that performance improvements followed a predictable power-law curve. Double the compute or data, and the model would reliably perform better. This consistency turned scaling into a formula, and the industry adopted it as gospel.
But scaling laws aren’t fundamental truths of the universe. Amodei compared them to Moore’s Law, which described the doubling of transistor density every two years and was used for decades to predict the evolution of computer power. Moore’s Law wasn’t a law of physics but an observation of industry trends driven by human innovation. When physical and economic limits hit, progress slowed. Similarly, scaling laws depend on practical realities—like access to high-quality data and compute—and those realities are becoming constraints.
The Golden Era of Scaling
For most of the last decade, scaling laws delivered incredible returns. Models like GPT-3 got better at predicting the next word in a sentence but soon enough they started performing tasks that were never explicitly programmed, like writing essays or debugging code. This was the magic of scaling—unexpected capabilities emerging from simply making the model bigger and training it longer.
Why did scaling work so well for large language models (LLMs)? It’s partly because language itself is hierarchical and statistical. At every level—words, sentences, paragraphs—patterns exist that models can learn. Small models capture basic grammar and syntax; large models grasp abstract reasoning and context. Combined with the architecture of transformers, which efficiently process sequences of data, this made scaling not just effective but transformative.
However, the success of scaling relied on three key assumptions:
1. Unlimited High-Quality Data: As the internet provided massive datasets, models improved by ingesting more diverse and complex text.
2. Compute Efficiency: Advances in hardware (GPUs and TPUs) made it feasible to train larger models on bigger datasets.
3. Generalizability: Larger models didn’t just perform well on one task—they excelled across tasks without the need for retraining.
For years, these conditions held, fueling exponential progress. But now, cracks are beginning to show.
Cracks in the Scaling Paradigm
Scaling laws have reached a point where the returns on investment—whether in terms of compute, data, or model size—are no longer as significant as they once were. This decline is driven by several factors:
1. Data Exhaustion: High-quality text data from the internet has largely been consumed. Newer models are increasingly trained on synthetic or lower-quality data, which raises concerns about the limits of generalization and the propagation of errors in downstream tasks.
2. Emergent Risks: As models scale, they exhibit unexpected emergent behaviors—capabilities that weren’t explicitly trained but arise from the size and complexity of the model. While some emergent behaviors are beneficial, others introduce risks, such as misalignment or unanticipated biases.
3. Diminishing Returns: Larger models no longer guarantee proportionate performance improvements. The relationship between scale and capability has begun to plateau in several benchmarks, particularly in complex reasoning tasks. Again, this is an area we do not fully understand, in other words we can’t really say why this is happening. But we’re seeing this happening in different metrics we track to assess model’s improvement.
These cracks don’t mean the end of AI progress, but they do signal the need for new paradigms. If scaling laws are no longer the cheat code they once were, researchers must look toward alternative methods—like better architectures, domain-specific models, and enhanced interpretability techniques—to drive the next wave of innovation.
Are We Close to AGI?
Amodei suggested that AGI—defined as achieving Nobel Prize-level intelligence across various domains—could emerge as soon as 2026. Elon Musk echoed a similar timeline, predicting that AI will surpass human intelligence in just a few years. These predictions are grounded in the remarkable pace of AI advancements and the growing capabilities of current systems.
However, not everyone agrees. Critics like Yann LeCun argue that while progress is significant, we’re still decades away from AGI. LeCun points out that AI systems today fall short of even basic animal intelligence in many respects. This divergence of opinions underscores how complex and unpredictable the path to AGI remains. But if the optimists are correct, we are far closer to AGI than most of us might realize—making discussions about alignment, safety, and societal implications all the more urgent.
One trivial but good point I read against the fact that AGI is very close is the disfunctional organizational movements at OpenAI. Some people argue that the fact that the top leadership of OpenAI (which should be among the leaders in this race to AGI) are leaving the company signals we’re still not as close as some people think. Why would you leave the leading company a moment before the discovery you have invested all your life into? Without even mentioning the financial implication of leaving before cashing in the benefits of being in the company that launches AGI… to a simple man like me, this was probably the most convincing argument!
AI Ethics is more practical than I thought!
In the second part of the podcast, Lex interviews Amanda Askell, the head of Anthropic’s ethics team, who does potentially the coolest and scariest job in the world: she shapes Claude “characther”. Basically her team is in charge of forging the ethical profile of AI. Most of her team’s work is around answering the question “why did Claude say what it just said?”.
She was really interesting to hear, and frankly I am so happy that these considerate and rational scientists and researchers are in charge of these potentially life-changing developments.
The one surprising and refreshing thing for me to hear is that defining AI ethics is a lot more about engineering than it is about “philosophy and moral talks”. Askell described her approach as deeply empirical, relying on tools like Reinforcement Learning from Human Feedback (RLHF). In this method, human evaluators score model outputs to create a feedback loop, iteratively aligning the system’s behavior with desired norms. For example, RLHF can train models to avoid biased or harmful outputs by incorporating real-world user feedback.
I really like how she explains that their role is to “nudge” Claude in a certain direction rather than “instructing him”. It’s really like educating a child, in many senses.She noted that ethical principles like fairness or harm reduction guide the work, but the actual execution is a matter of testing, measuring, and optimizing performance.
Mechanistic Interpretability: Understanding the Neural Unknown
Final part of the podcast is with Chris Olah, Anthropic’s lead researcher in mechanistic interpretability, a field focused on decoding how neural networks work. While AI systems like GPT deliver remarkable results, researchers often don’t fully understand why. And so there are full teams dedicated to understanding better how the models work, and how they are so efficient. At the end of the day this is not very different vs a neuroscientist trying to understand how human brain works.
Olah described his work as opening a “black box” to map how different neurons, layers, and activations contribute to specific behaviors. For example, his team has identified neurons linked to concepts like sentence syntax or object recognition. Most of his team’s work is around answering the question “how did AI do what it just did?”.
Conversely to the previous section, this work sounded a lot more abstract than I had anticipated. Unlike fine-tuning or optimization, this work resembles theoretical scientific research. Olah emphasized the importance of forming bold hypotheses and rigorously testing them, even if they’re wrong. He likened the process to early astronomy, where inaccurate star maps laid the groundwork for modern celestial mechanics. Similarly, today’s interpretability research is building foundational insights into neural networks.
Really fascinating how little we still understand of AI, sounds to me like nature or evolution: we know the basic rules and have some heuristic of how it develops (the scaling laws), but there’s still so much unknown!
I hope you enjoyed the read, wish you all a fantastic weekend!
Giovanni
Book in a tweet
In a post about Artificial Intelligence, the reading recommendation can’t not be about Human Stupidity!
A serious economist, Cipolla, argues that “stupidity is the most destructive force in society: it’s universal, irrational, and costly to everyone—yet often underestimated”.
Quick but really brilliant book
