The Social Scaling Law: What Did Ilya See
The Curious Case of Hominid Brain Development
During Ilya Sutskever’s NeurIPS 2024 presentation, he highlighted one of evolution’s most fascinating puzzles: the extraordinary trajectory of hominid brain development. In nature, we observe a remarkably consistent pattern known as Kleiber’s law, where brain size scales with body mass. This scaling law holds true across an impressive range of species, from tiny shrews to massive elephants.
Yet hominids — our ancestral lineage — break this universal pattern in a dramatic fashion. While a chimpanzee’s brain size follows the expected scaling law, early human ancestors like Homo habilis and subsequent species show a striking deviation, with brain sizes 2–3 times larger than what the standard mammalian scaling would predict. This isn’t a subtle statistical anomaly; it’s a revolutionary shift in evolutionary trajectory.
This extraordinary departure from one of nature’s most consistent patterns raises profound questions: What selective pressures were so powerful that they could override a scaling law that had held steady for hundreds of millions of years? What made our ancestral line so unique that it demanded such exceptional cognitive capacity?
The Social Brain Hypothesis
The key to understanding our exceptional brain development may lie in what’s known as the social brain theory, a hypothesis that has gained significant traction in both evolutionary biology and, more recently, artificial intelligence research. The theory, thoroughly explored in “A Brief History of Intelligence,” which is a book I am reading recently, presents a compelling explanation for our cognitive evolution.
At its core, the theory identifies a remarkable pattern: the neocortex ratio — a crucial measure of brain development — shows a strong positive correlation with social group size in primates. What makes this correlation particularly intriguing is its specificity; it doesn’t hold for other highly social animals. Buffalos, for instance, live in herds of thousands, yet their brain development follows standard mammalian scaling. This suggests something uniquely demanding about primate social structures.
The critical distinction lies in the nature of social interactions. Primate groups aren’t merely collections of individuals — they’re complex social networks where each member must navigate intricate relationships and power dynamics. This requires sophisticated “theory of mind” capabilities: the ability to not only recognize others’ current mental states but to predict their future actions and understand their knowledge gaps.
This social complexity manifests in several key behaviors that create a cognitive arms race:
- Strategic Alliance Formation: Primates must remember past interactions, evaluate potential partners, and maintain multiple relationships simultaneously.
- Complex Deception: The ability to mislead others while avoiding being misled, requiring multi-level thinking about other’s beliefs.
- Power Dynamics: Continuous monitoring and adjustment of social status, requiring understanding of hierarchy and influence.
- Social Network Navigation: Managing relationships not just with immediate contacts but understanding and influencing extended social networks.
As Ilya Sutskever highlighted in his 2017 talks, this “social life incentivizes evolution of intelligence” hypothesis offers a compelling explanation for our cognitive development. The constant pressure to outmaneuver social competitors while maintaining beneficial alliances creates an evolutionary environment that continuously rewards increased cognitive capacity.
Drawing Parallels with AI Development
The social brain hypothesis offers more than just evolutionary insights — it provides a compelling blueprint for artificial intelligence development through self-play systems. Just as primate social competition drove cognitive evolution, self-play creates an artificial environment where agents must constantly adapt and improve to outperform their competitors (often versions of themselves).
The success of this approach has been remarkable:
- AlphaZero revolutionized chess by developing strategies through pure self-play, without any human gameplay data.
- OpenAI’s DOTA2 agents learned to coordinate in teams and develop sophisticated strategies that surprised professional players.
- MuZero demonstrated that self-play could work even without knowledge of the environment’s rules.
What makes these achievements particularly significant is the rate of improvement. Both Ilya Sutskever and Demis Hassabis have highlighted a crucial observation: self-play systems often show rapid, sometimes discontinuous improvements in capability.
Here is what Demis talk about the speed of the intelligence development:
I remember this day very clearly, where you sort of sit down with the system starting off random. In the morning, you go for a cup of coffee, you come back. I can still just about beat it by lunchtime, maybe just about. And then you let it go for another four hours. And by dinner, it’s the greatest chess-playing entity that’s ever existed. And it’s quite amazing, looking at that live on something that you know well, like chess, and you’re expert in and actually just seeing that in front of your eyes. And then you extrapolate to what it could then do in science or something else, which of course, games were only a means to an end.
This mirrors the “cognitive arms race” we see in primate evolution, where social competition drives accelerated development of increasingly sophisticated strategies.
The implications for AI development are profound. As Sutskever noted at NeurIPS 2024, while internet-based training data is finite and may eventually be exhausted, self-play represents a renewable resource for AI development. It effectively transforms computational power into training data through the generation of novel interactions and scenarios. More compute = More data
Current scaling laws suggest that increased compute and data consistently yield better performance, making self-play a potentially infinite source of improvement.
This creates an interesting parallel: just as social competition generated increasing cognitive complexity in primates, self-play might offer a path to developing more sophisticated AI systems. The key advantage is that self-play:
- Generates novel situations beyond the training data.
- Forces adaptation to increasingly sophisticated strategies.
- Creates pressure for meta-learning and strategic thinking.
- Develops transferable skills that work across different contexts.
From Theory to Evidence: Self-Play’s Creative Potential
While the social brain theory offers compelling theoretical insights, the 2017 paper “Emergent Complexity via Multi-Agent Competition” [Bansal, et al., 2017] which Ilya also involved provides concrete evidence that self-play can generate genuine novelty and complexity. This is particularly significant in the context of current AI debates about “stochastic parrots” — the criticism that large language models merely recombine existing patterns without true understanding or creativity.
AlphaGo’s famous “Move 37” against Lee Sedol demonstrated how self-play systems can develop strategies that transcend human knowledge and convention. This wasn’t just a statistical anomaly or a recombination of existing patterns — it was a genuinely novel approach that emerged from the competitive dynamics of self-play.
The paper demonstrates this emergence of complexity through several key observations:
- Deception Strategies: Agents learned to fake intentions and mislead opponents — behavior that wasn’t explicitly programmed but emerged from competitive pressure.
- Transferable Skills: Techniques developed in one competitive scenario proved useful in different contexts, suggesting genuine learning rather than mere pattern matching.
- Complex Behaviors from Simple Rules: The environment specified only basic rewards, yet the interaction between competing agents led to sophisticated strategic thinking.
What makes this particularly interesting is the mechanism of complexity generation. Unlike current LLMs that work with a fixed dataset of human-generated content, self-play systems create novel scenarios through agent interaction. This suggests a potential path beyond the current scaling laws and their eventual limitations. While LLMs might reach the boundaries of available human-generated data, self-play systems generate their own novel training scenarios through agent interaction. This could provide a renewable source of genuinely new patterns and strategies, rather than just recombinations of existing data.
Looking Forward: The Next Scaling Law
The parallels between primate brain evolution and AI development through self-play point toward an intriguing future direction for artificial intelligence. While current Large Language Models and RLHF have shown remarkable progress, the social brain hypothesis suggests we might be just scratching the surface of possible AI advancement.
“What is the right thing to scale?” I think that Ilya proposed an interesting research question. Let’s go back to what worked for hominids. The paper “Evolution in the Social Brain” revealed a crucial insight: neocortex ratio scales predictably with group size in primates. This suggests that cognitive complexity isn’t just about individual processing power — it’s about the capacity to handle increasingly complex social interactions. This insight might offer a blueprint for the next phase of AI scaling laws.
Current scaling laws focus primarily on model size, compute, and dataset size. But what if, like in primate evolution, the key scaling factor is actually the complexity of social interaction? This suggests several potential directions for future research:
- Multi-Agent Scaling: Instead of just scaling model size, we might need to scale the number of interacting agents.
- Interaction Complexity: Developing environments that foster increasingly sophisticated social dynamics.
- Competitive Pressure: Creating scenarios where agents must constantly adapt to each other’s strategies.
- Meta-Learning: Encouraging agents to develop general principles from specific interactions.
The evidence from both evolutionary history and current AI experiments (AlphaZero, DOTA2) suggests that competitive social interaction might be a crucial driver of intelligence. Just as our ancestors’ cognitive capabilities were shaped by increasingly complex social dynamics, future AI systems might advance through carefully designed multi-agent interactions.
Key Research Directions
The path forward might involve:
- Developing new metrics for measuring multi-agent system complexity.
- Creating scalable environments that can support large numbers of interacting agents.
- Understanding the relationship between agent population size and emergent behavior complexity.
- Designing reward structures that encourage both competition and cooperation.
The social brain theory provides not just an explanation of our past evolution, but potentially a roadmap for future AI development. As we approach the limits of current scaling laws based on static datasets, the renewable complexity generated by multi-agent interactions might offer the next frontier in artificial intelligence.
Citations:
Bansal, Trapit et al. “Emergent Complexity via Multi-Agent Competition.” ArXiv abs/1710.03748 (2017): n. pag.
Dunbar, R I M, and Susanne Shultz. “Evolution in the social brain.” Science (New York, N.Y.) vol. 317,5843 (2007): 1344–7. doi:10.1126/science.1145463
Manger, Paul R et al. “The evolutions of large brain size in mammals: the ‘over-700-gram club quartet’.” Brain, behavior and evolution vol. 82,1 (2013): 68–78. doi:10.1159/000352056
