The recent TED Talk by Noam Brown, a research scientist at OpenAI, sheds light on the future of AI and its potential to evolve beyond current limitations. He discusses how traditional AI models have relied heavily on scaling data and computing power, but introduces a new approach that mimics human reasoning. This shift could lead to significant advancements in AI capabilities.
Key Takeaways
- AI progress has been driven by scaling data and compute power.
- Traditional models may soon hit a plateau due to rising costs.
- Introducing slower, more deliberate reasoning can enhance AI performance.
- The new o1 model from OpenAI focuses on this type of reasoning.
The Current State Of AI
In the last five years, the growth of AI has been remarkable, primarily due to scale. While there have been some algorithmic improvements, the core architecture of models has remained largely unchanged since 2017. The main difference? The sheer amount of data and computing power used to train these models has increased dramatically.
For instance, training GPT-2 in 2019 cost around $5,000. Fast forward to today, and training frontier models can run into the hundreds of millions of dollars. This raises a critical question: will we soon reach a point where AI development plateaus due to these escalating costs?
The Poker Experiment
To illustrate his point, Noam shares a personal story from his PhD days, where he worked on AI that could play poker. Initially, the research community believed that simply scaling up the models would lead to better performance. They trained their poker AI on nearly a trillion hands, but when it faced off against top human players, it lost badly.
What was fascinating, however, was the difference in decision-making speed. The AI made decisions in about ten milliseconds, while human players took their time, sometimes thinking for several minutes on tough decisions. This led Noam to consider the concept of System 1 and System 2 thinking, as described by Daniel Kahneman.
- System 1: Fast, intuitive thinking (like recognising a face).
- System 2: Slower, more deliberate thinking (like solving a complex problem).
The Power Of Thinking Time
After the competition, Noam conducted experiments to see how much time spent thinking could improve the AI's performance. The results were astonishing: allowing the AI to think for just 20 seconds during a hand of poker provided the same performance boost as scaling the model size and training time by 100,000 times. This was a game-changer.
Realising the potential of System 2 thinking, the team redesigned their poker AI to incorporate this approach. In a subsequent competition, they not only improved their strategy but also won decisively against the best human players.
Lessons From Other Games
This principle of taking time to think isn't limited to poker. Similar patterns have been observed in other games like chess and Go. For example, IBM's Deep Blue and DeepMind's AlphaGo both took time to think before making moves, which significantly enhanced their performance.
The Future With o1
Now, with the introduction of OpenAI's new o1 model, the focus is on this slower, more thoughtful approach. The o1 model allows for varying thinking times based on the complexity of the question. This opens up a new avenue for AI development, moving beyond just scaling up training data.
The cost of querying these models is still low, but the potential for improved performance by allowing more thinking time could be worth the investment. For critical problems, like medical research or complex scientific inquiries, the benefits of a more thoughtful AI could far outweigh the costs.
Conclusion
As Noam Brown concludes, the evolution of AI is not a distant future; it’s happening now. The shift towards incorporating System 2 thinking into AI models could lead to breakthroughs we can’t yet imagine. While some may still believe AI will plateau, the evidence suggests otherwise. The future of AI is bright, and it’s all about giving it the time to think.