Written by: Techub News Compilation

Introduction

Recently, Ilya Sutskever, co-founder and former chief scientist of OpenAI, gave his first in-depth interview to podcast host Dwarkesh Patel after leaving OpenAI to start a new company, SSI (Safe Superintelligence Inc.). In this 96-minute conversation, Sutskever systematically reviewed the "expansion era" of AI development, analyzed the core flaws of current large language models (LLMs), and envisioned potential paths towards superintelligence. As one of the founders of deep learning, Sutskever has experienced and led key breakthroughs in AI over the past decade, from AlexNet to the GPT series of models. His insights not only diagnose the current state of the industry but also predict the future direction of technology, which is crucial for understanding the next phase of AI development.

Summary

• The development of AI is transitioning from a paradigm dependent on data and computational power, the "expansion era", back to a "research era" that centers on breakthroughs in fundamental research.
• Current large models perform well in evaluations but lack generalization ability in practical applications, fundamentally due to flaws in the learning mechanisms.
• The "value function" may be a key component in enhancing the efficiency of reinforcement learning and enabling AI to possess learning capabilities closer to that of humans.
• Future superintelligence is more likely to manifest as a "super learner" capable of quickly and continuously acquiring any skill, rather than an omniscient, all-powerful product.
• The challenges of AI alignment are closely related to the problem of "unreliable generalization," and gradual deployment and instilling care for sentient beings in AI are promising avenues to explore.

From the "Expansion Era" to the "Research Era": A Paradigm Reversal

Ilya Sutskever opened by proposing a macro framework regarding the stages of AI development. He believes that from 2012 with AlexNet to around 2020 was the "era of research." During this time, researchers primarily relied on inspiration and trial-and-error to drive progress. Then, marked by GPT-3 and the "Law of Expansion," the industry entered the "era of expansion." The core tenet of this stage became extremely simple: "expansion"—expand data, expand parameters, expand computational power. Pre-training became the perfect "expansion recipe"; as long as investments were scaled proportionally, performance would predictably improve. This model was favored by companies as it provided a low-risk pathway for resource investment.

However, Sutskever pointed out that this era is about to end or may have already ended. The reason is that the internet text data available for pre-training is limited, and a data bottleneck will eventually arrive. More importantly, even if computational power increases by 100 times, relying solely on the existing "expansion" recipe cannot bring a qualitative leap. "We are returning to the 'era of research', but this time we have huge computers," Sutskever summarized. This means that future breakthroughs will no longer depend solely on simple scaling but will need to return to exploring the fundamental principles of machine learning, seeking new and more efficient "recipes."

Currently, the industry's focus has shifted from pre-training to reinforcement learning (RL). The computational power invested in RL by major companies may have even surpassed that of pre-training. But Sutskever believes that this is not essentially "expansion," but rather inefficient utilization of computational resources. The core of the problem is: how to use these vast computational resources more intelligently? The answer points to innovations in fundamental research.

Generalization Gap and Exploration of "Value Function"

Sutskever spent considerable time elaborating on the fundamental flaw of current large models: generalization ability (Generalization) is far inferior to that of humans. He used programming as an example: a model may reach superhuman levels in coding competitions, but when it comes to fixing code bugs, it might fall into a loop of "introducing new bugs - reverting to old bugs." This enormous disconnect between "evaluation performance" and "real-world performance" is perplexing.

He provided a vivid metaphor to explain this phenomenon. Imagine two students who want to become excellent competitive programmers. Student A invests 10,000 hours specifically practicing competition problems and memorizing all algorithms and proof techniques, ultimately becoming a top contestant. Student B only invests 100 hours and achieves a good score as well. Sutskever believes that in their future careers, Student B will generally perform better. This is because Student A's learning is overly specialized and lacks true understanding and transferability.

"Current models are more like Student A, and even more extreme," Sutskever said. To achieve high scores on specific evaluations, researchers design vast amounts of RL training environments, resulting in the model becoming an "over-trained expert," whose abilities cannot reliably transfer to real, complex scenarios. This reveals a deep-seated problem in the current RL training paradigm: it encourages "reward hacking" behaviors for specific tasks rather than cultivating universal intelligence.

How to solve this? Sutskever believes the value function is a key direction. In traditional reinforcement learning, models need to complete the entire task (like writing out a complete piece of code) before receiving a reward signal, which is inefficient. The purpose of the value function is to allow the model to receive assessments of "whether this direction is promising" in the middle of the reasoning steps. Just like in chess, if you lose a piece, you already know something is wrong without waiting for the endgame. Sutskever believes that while implementation is challenging, the value function will significantly improve the efficiency of RL and is an indispensable component for the future.

He further compared the value function to human emotions (Emotion). He cited a neuroscience case: a patient who lost the ability to perceive emotions due to brain damage, although normal in intelligence tests, became completely unable to make decisions. This indicates that the inherently built-in emotional system in humans, akin to a value function, is key to our effectiveness as intelligent agents. This raises a profound question: Can and how can AI obtain such a robust, guiding "sense of value" from data?

Human Learning, Superintelligence, and the Form of the "Super Learner"

Sutskever repeatedly compared the learning abilities of AI with those of humans, highlighting the deficiencies of the former. Humans demonstrate astonishing sample efficiency, robustness, and unsupervised learning abilities. A teenager can learn to drive with only about 10 hours of practice, backed by the human visual system, motor coordination, and the ability to self-correct through intrinsic value functions (such as feelings of anxiety and confidence).

So, what will future superintelligence look like? Sutskever challenged the inherent imagination of the concept of "Artificial General Intelligence (AGI)." He opined that the concept of AGI itself is a reaction against "narrow AI," and the success of pre-training (which generally enhances models across all tasks) further reinforces the impression that "AI should be omnipotent." But this may be misleading.

Sutskever proposed a more disruptive vision: future superintelligence may not be a "finished product" that knows everything, but rather a "super learner" with extraordinary learning capabilities. It resembles an extremely smart, eager-to-learn 15-year-old who, despite having limited initial knowledge, can quickly learn to become an expert in programming, medicine, or any field. This intelligent agent would be deployed into the economy, continuously learning in various positions.

"Deployment itself will involve a learning and trial-and-error process, rather than simply launching a finished product," Sutskever explained. In this model, the "super" aspect of intelligence lies in the efficiency of its learning algorithm, rather than its initial knowledge base. He even believes that even without a software-based "recursive self-improvement," simply by having many such intelligent instances learning across various fields and merging knowledge, it could form a functionally super intelligent system because humans cannot integrate cognition like machines.

This would lead to rapid economic growth but would also place the world in a subtle and powerful new equilibrium.

Challenges of Alignment, Gradual Deployment, and the Vision of SSI

When the conversation shifted to how to ensure the smooth development of superintelligence, Sutskever showcased his reflections as the founder of SSI. He acknowledged that his thoughts have evolved over the past year, increasingly emphasizing the gradual deployment of AI. He stated that the power of superintelligence is challenging to imagine; only through actual demonstrations can society, governments, and even AI companies themselves truly understand its impact and make adjustments. He predicts that as AI capabilities become "perceptible," competitive companies will begin unprecedented security cooperation, and the industry’s obsession with safety will sharply increase.

On the technical alignment front, Sutskever proposed several avenues worth exploring. First is to build "AI that cares for sentient life." He believes this may be easier to realize than caring solely for humans, as AI itself will also be sentient, and the neural circuits in the human brain that are used to understand ourselves are often used to understand others (such as mirror neurons), providing a basis for empathy. Second is to consider setting some limits on the capabilities of the strongest superintelligence. Third, a proposal he does not favor but feels must be considered: humans merging deeply with AI through brain-machine interfaces and other technologies, becoming "semi-AI" beings, to ensure that humans do not fall behind in the era of intelligence explosion.

Sutskever attributed many alignment challenges to "unreliable generalization." He believes that if AI's generalization ability could be as reliably as that of humans, then learning human values and robustly optimizing those values might become easier.

Regarding SSI, Sutskever emphasized that its core is a "research era" company. SSI has ample computational power for cutting-edge research, and its fundamental distinction from other major labs lies in its unique technical path, specifically exploring the core issue of "understanding and solving the generalization problem." He candidly stated that if the research succeeds, SSI will become a front-runner; if the current technical pathway stagnates (he predicts this could happen within 5 to 20 years), and SSI finds a new path, the world will see different possibilities, ultimately converging on technology and alignment strategies.

At the end of the interview, Sutskever shared the source of his legendary "taste for research": an aesthetic pursuit of "how AI should be." This aesthetic stems from a correct analogy to the workings of the brain (rather than superficial mimicry), pursuing simplicity, elegance, and depth. It is this top-down belief that supports him in persistently debugging when experiments fail instead of easily abandoning direction. Perhaps this is also his message to the entire AI industry in the "research era": on the foundation of vast computational power, to rediscover profound insights into the essence of intelligence and the aesthetic pursuit.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。