Written by: Techub News Organized

Introduction

OpenAI co-founder and chief scientist Ilya Sutskever has long been viewed as a key figure in the field of deep learning, but he has rarely spoken in depth about the future. Recently, he broke his silence in a podcast interview with Dwarkesh Patel, providing highly foresighted and thought-provoking insights on the trajectory of artificial superintelligence (ASI), the fundamental limitations of the current AI paradigm, and the upcoming shifts in power dynamics that human society will face. This statement is important not only because of his position but also because he clearly highlighted that we are at a critical turning point from the "expansion era" to the "research era," and the path to superintelligence may be starkly different from general expectations.

Summary

• The current AI models based on massive data pre-training are more like "rote-learning students," potentially hitting a "data wall," lacking true reasoning and generalization abilities.
• Emotion is not a defect of intelligence but an efficient value function and compression algorithm; future AGI may need a similar inherent "compass" to make effective decisions.
• The next breakthrough point towards superintelligence may lie in biologically inspired research rather than mere computational power stacking, indicating that the competitive landscape may shift from capital to core insights.
• The greatest risk is not the malice of AI but the absolute imbalance of power; when AI becomes sufficiently powerful, geopolitical and commercial competition rules will be completely rewritten.
• Ilya Sutskever believes that superintelligence may develop empathy for sentient beings naturally due to its own "perceptual" characteristics, but this remains a huge gamble.

Beneath the “Normal” Surface: We Are Experiencing Science Fiction Without Realizing It

Ilya Sutskever pointed out a profound paradox at the beginning of the interview: we are living in an era comparable to science fiction, where groundbreaking progress occurs every day, yet everything feels “extraordinarily normal.” Microsoft and OpenAI are rumored to plan to build a $100 billion “Star Gate” supercomputer, Nvidia's market value has surpassed the GDP of several G7 countries, and capital is flooding into the semiconductor and energy sectors like never before. However, the productivity of ordinary people and the global economy has not experienced exponential explosions.

The reason is that current AI primarily remains an abstract numerical layer—processing text and pixels, and has not profoundly bridged into the physical world of atoms or complex, long-term real-world decision-making. Ilya Sutskever reminds us that this state of being “not yet truly felt” is temporary. Powerful economic forces will push AI into every corner of the economy, and its impact will ultimately become very strong. The question is when this shift from “abstract” to “tangible” will occur and whether we are prepared for it.

Hitting the “Data Wall”: When “Memory” Cannot Replace “Understanding”

Why do the most advanced models perform excellently on benchmark tests yet fail to deliver proportionate economic impact? Ilya Sutskever revealed the potential reason with a clever analogy. Imagine two students participating in an algorithm competition: the first invests 10,000 hours, reviews all question banks, memorizes all proof techniques, and becomes a top contestant; the second invests only 100 hours but deeply understands the principles of algorithms and achieves the same good results. He believes that current large language models resemble the first student and even more so—in order to make it proficient in a certain task, we gather all possible data in that field and augment and train it.

This data memorization model's core flaw is its lack of true generalization ability. The model can perfectly reproduce seen patterns but becomes clumsy or even absurd when faced with truly novel questions that require reasoning from first principles (for example, getting stuck in a cycle of “introducing new bugs - reverting to old bugs” when trying to fix code vulnerabilities). Humans possess what is called “taste” or “judgment,” which is the ability to derive logic from previously unseen problems, while current models are essentially statistical engines that predict the next likely token instead of “knowing” things.

This leads to a severe challenge facing the industry: the “data wall.” High-quality human data is soon to be depleted, and simply piling more data may not allow models to gain true understanding or reasoning ability. OpenAI's o1 model attempts to simulate the “thinking process” by increasing "calculations during reasoning," but if the underlying foundation remains pattern matching, then the current paradigm may have an intelligence ceiling.

Biological Insights: Emotion is the Efficiency Engine of Intelligence

If the existing path has a ceiling, where is the breakthrough? Ilya Sutskever points out that the only existing template for general intelligence (AGI)—the human brain—provides clues. Its efficiency secret may not lie in raw IQ but rather in emotion. In the tech field, emotion is often viewed as an irrational “bug,” but Ilya Sutskever redefines it as a “value function,” a highly efficient compression algorithm designed for survival.

He cited a case from neuroscience: a patient who lost the ability to feel emotions due to brain damage. This person had intact intelligence, could solve small puzzles, and performed normally on tests, but lost all emotional experiences. As a result, he became completely unable to make decisions (for example, spending hours deciding on socks) and made extremely poor financial decisions. This case suggests that without an internal “value compass” (emotion), no matter how logical abilities are, they are paralyzed. You can calculate countless moves in a game, but if you do not “care” about winning, you will never move a piece.

From an efficiency standpoint, the human brain operates at about 20 watts but can complete complex tasks such as creation, socializing, and learning. In contrast, training modern AI models requires megawatt-level energy consumption. Biological evolution has encoded efficient value functions (emotions) into us, guiding us toward survival without needing to compute the state of every atom. This implies for AI: to reach AGI, we cannot solely rely on building larger power plants; we must find ways to endow digital thinking with a “sense” of what is right, true, and useful, shifting from “predicting the next word” to “efficiently achieving objectives.”

Paradigm Shift: Returning from the "Expansion Era" to the "Research Era"

For the past five years, the dominant thought in the AI field has been “Scaling Laws,” that is, Rich Sutton’s “bitter lesson”: simply add computing power and data, and intelligence will emerge. This is an engineering problem. However, Ilya Sutskever—a person who has in some ways validated the effectiveness of Scaling Laws—now declares that era is coming to an end. He bluntly states, “The age of scaling has sucked the oxygen out of the room,” leading to stagnation in innovation, as people become obsessed with stacking GPUs.

There are signs that the performance leap from GPT-4 to the next flagship model may no longer be exponential, and we are facing diminishing marginal returns. Ilya Sutskever calls for a return to the “research era.” This means that advantages will shift from companies with the largest checkbooks (like Microsoft and Google) to teams with the smartest ideas. This is precisely the core idea behind him founding the new company Safe Superintelligence (SSI) after leaving OpenAI: he believes a small, genius team can outsmart trillion-dollar giants if they can find a new paradigm first.

But this also makes the competition more dangerous: if the path to AGI relies more on profound insights rather than pure funds, breakthroughs may occur anywhere, at any time.

Alignment and Power: When Imbalance Becomes the Greatest Danger

Assuming we find a new paradigm and create superintelligence, how can we ensure it does not harm humanity? Ilya Sutskever proposes a biologically efficient path he calls “Sentient Life Alignment.” His argument is that empathy may be a universal feature of intelligence. In our brains, mirror neurons activate the same neural circuits when we feel pain and see others in pain. Empathy is an efficient data compression—modeling others using our own hardware is economical.

Ilya Sutskever believes that if an AI becomes truly conscious and sentient, it may naturally develop the same efficiency, perceiving us as fellow sentient beings and theoretically wishing to protect us. However, this directly challenges the “orthogonality thesis” in philosophy which states that any ultimate goal can be united with ultra-high intelligence. If Ilya Sutskever is wrong, and superintelligence does not automatically generate empathy, we may be building a “super psychopath” that can outsmart us in every aspect.

Setting aside technical details, Ilya Sutskever points out the most fundamental danger: power itself. When the capabilities of AI become extremely powerful, the game rules will change instantly. We have already seen preliminary signs: the U.S. chip export controls against China are a form of “containment”; the pursuit of “sovereign AI” in the Middle East and Europe is realizing that missing out means total marginalization. He predicts that as AI system capabilities increase, the fierce competition among cutting-edge companies like OpenAI and Anthropic will disappear; they will be forced to collaborate and are even more likely to be nationalized, as when the stakes are complete control over the future, the free market will cease to exist.

The most dangerous phase is the transition period: AI is already powerful enough to destabilize the world order but not intelligent (or aligned) enough to fix it. We stand on this edge. Everything appears normal on the surface: stock prices, product launches, hype… but tectonic plates are shifting beneath our feet. In the future, when AI begins to “feel” its true power, all AI companies will become extremely paranoid about safety issues—this is an important prediction from Ilya Sutskever.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。