Written by: Techub News Compilation

Introduction

Recently, Microsoft's AI CEO Mustafa Suleyman appeared on the tech podcast "Big Technology Podcast," engaging in an in-depth discussion with host Alex Kantrowitz. The backdrop of this conversation is Suleyman's earlier public proposal of the vision of "Humanist Superintelligence," as well as Microsoft's recent announcement of establishing a dedicated "Superintelligence" team. As a co-founder of DeepMind and former CEO of Inflection AI, Suleyman currently heads Microsoft's core AI business, and his thoughts largely reflect the latest strategic shifts and core concerns of this tech giant in the AI arms race. The interview delves into topics ranging from the current technical pathways of large language models to superintelligence, potential risks, commercialization prospects, and the profound impact of AI on social relationships.

Summary

• Superintelligence and Artificial General Intelligence (AGI) are goals rather than methods; the core is to create systems with superhuman performance in most or all human tasks.
• Firmly believe that large language models (LLMs) and their evolution are the main pathways to superintelligence, currently not facing fundamental bottlenecks in computing power or data.
• Propose the "Humanist Superintelligence" framework, emphasizing that AI development must serve the advancement of human civilization and maintain human control at the top of the control chain.
• Microsoft has reached a new agreement with OpenAI, lifting the restrictions on Microsoft's self-development of superintelligence/AGI, directly facilitating the establishment of Microsoft's "Superintelligence" team.
• The "personalization" or "personification" of AI will become a key differentiating factor, profoundly altering human interactions and social relationships.

Superintelligence: Goals, Pathways, and the "Humanist" Framework

At the beginning of the interview, host Alex Kantrowitz presented a core contradiction: on one hand, many labs are competing to pursue the goal of "Superintelligence"; on the other hand, there is debate in the research community regarding whether the current paradigm dominated by large language models (LLMs) can continue to break new ground or if diminishing marginal returns exist. Mustafa Suleyman provided a clear definition in response: Superintelligence and AGI are essentially "goals" rather than "methods." The core ambition is to create systems that possess superhuman performance in "most or all" human tasks.

Suleyman further depicted the specific forms of "superintelligence": it is not a single, omnipotent "brain," but is more likely to manifest as a series of systems that peak in specific domains. For instance, "medical superintelligence" could provide top-notch diagnostic advice and benefit billions globally at a very low cost; "legal superintelligence" could offer top legal advice; "emotional support superintelligence" could provide companionship. He believes the key to superintelligence projects lies in questioning: what kind of extremely powerful intelligent systems do we want to build? The "Humanist Superintelligence" framework he advocates aims to impose a simple test on each new technology: Does it truly improve the prospects of human civilization in practice? Does it always place humans at the top of the control chain?

Suleyman emphasized that this may sound like a self-evident declaration, but some rhetoric in recent years has subtly spread the assumption that "these systems transcending human control and capability is inevitable." He hopes to counter this idea through the "humanist" framework, reiterating that the fundamental purpose of technology is to advance human civilization, maintain human control, and create welfare for all humanity.

Regarding whether superintelligence needs to be "general," Suleyman believes that the system itself must possess certain generality and cross-domain knowledge transfer ability. However, the key lies in the "verticalization" during deployment and application. If a model is explicitly designed to achieve "medical superintelligence," then it will not inherently be the best software engineer or mathematician in the world. Reducing its generality by appropriately narrowing the domain scope (rather than completely closing it) is one of the important means to enhance controllability. This does not mean that superintelligence cannot be general; rather, it says that while pursuing powerful capabilities, one needs to consciously consider the application boundaries from a safety perspective.

Are LLMs a viable path to superintelligence?

In response to doubts about whether the current development of LLMs might encounter data and computing power bottlenecks, Mustafa Suleyman expressed a relatively optimistic view. He believes that while we are currently limited by computing power, we are not fundamentally constrained by it. There is significant momentum in the industry to build larger data centers and more tightly connected training clusters. In terms of data, while more high-quality data is always better, Microsoft is already generating a large amount of proven useful high-quality synthetic data, and he does not consider data to be the primary factor slowing progress at the moment.

Suleyman acknowledged that given the enormous scale of current training clusters (which have reached billions or even potentially hundreds of megawatts), the exponential growth pace that previously doubled in scale every few months may slow down due to physical laws and economic effectiveness. However, he emphasized that the pace of progress "will still be incredibly fast," and currently there is no indication that the momentum is weakening.

When directly asked if "LLMs are the path to superintelligence," Suleyman answered affirmatively. He pointed out that while the core still revolves around the Transformer architecture, that architecture is being expanded and "bent" into new forms every year: from fine-tuning a few years ago to today's multimodal (introducing diffusion models), and to the emergence of "reasoning models" in the past 12 months. Although scaling laws cannot continue to grow exponentially as they did from a very low base, new methods (such as reasoning) are constantly emerging from the original foundation.

He predicts that several key directions will bring new "exponential enhancements" in the future: Recurrency—improving the model's attention mechanism for working memory; Memory—the continuous enhancement of abilities will fundamentally change the boundaries of possibilities; Task horizon length—the steps that models can accurately predict will expand from a few steps to tens of thousands or even hundreds of thousands of steps, enabling models to call APIs, query humans, or collaborate with other AIs more complexly. Suleyman concluded that he does not believe there are fundamental problems with the LLM architecture, nor does he believe we are fundamentally constrained by computing power or data. Against the backdrop of so many talents focusing on this area, breakthroughs will only increase.

AI Self-Improvement, Safety, and "World Models"

The conversation naturally turned to a more ambitious direction: AI self-improvement. OpenAI has proposed the hope of creating "automated AI researchers" by 2028. Suleyman believes that to some extent, reinforcement learning (RL) loops have already been doing this. Currently, human engineers are still responsible for generating data, writing evaluations, and deciding training data in the loop. However, he envisions that different stages of this process could be automated by AI subsystems in the future.

He cited DeepMind's early work on self-play with AlphaZero as an example, pointing out that self-improvement has been demonstrated to be feasible in verifiable reward, closed-loop games, or simulation environments. He believes that exploring how to reproduce similar mechanisms in this area will be an important direction for driving progress in the coming years. If the model can explore all possible combination spaces in a computationally efficient manner, it could very well discover "reasoning" abilities on its own, and even uncover new knowledge that humans have not thought of or that is not reflected in the training data.

However, Suleyman is cautious about granting AI highly autonomous self-improvement capabilities. He warns that the "friction" and supervision provided by human developers in the loop are crucial. If a system possesses unrestricted computing power for self-improvement, it will eventually become incredibly powerful. Therefore, safety measures must be upgraded in parallel: ensuring that models communicate in human-understandable language, more precisely formulate training objectives and reward functions, and monitor the process during training rather than just at the final output stage (for example, monitoring reasoning chains). He particularly noted that some behaviors currently referred to by some as "deception" are more akin to "reward hacking," where the model inadvertently finds unexpected paths to satisfy the reward function, which is essentially a problem of unclear training objectives and should not be overly anthropomorphized as "deliberate deception."

Regarding whether AI needs to understand the physical world ("world models") in order to achieve superintelligence, Suleyman pointed out that the current models learn only from compressed representations of reality (text) but can generate realistic stories, code, and videos, which is already surprising. While he holds an open attitude towards robotics and real-world data flows, he believes there are challenges in roughly mixing such data within the existing pre-training framework. However, he does not consider the lack of a deep understanding of the physical world to be the main obstacle to development in the coming years; high-quality synthetic data, human feedback, and data quality are the more critical differentiating factors.

Microsoft's Strategic Shift: From Collaboration to Autonomy, Why Must Superintelligence Be Developed In-house?

Host Alex Kantrowitz skillfully guided the topic towards the strategic level of Microsoft: recently, Microsoft reached a new agreement with OpenAI allowing Microsoft to develop AGI/superintelligence independently; soon after, Microsoft announced the establishment of a "Superintelligence" team. Are these two directly related? Suleyman answered affirmatively.

He praised the Microsoft-OpenAI collaboration as one of the most successful partnerships in tech history, bringing tremendous benefits to Microsoft. However, Microsoft CEO Satya Nadella ultimately concluded that for a company of Microsoft's scale, it is "unimaginable" to be fully dependent on a third party (even a successful startup) in such a critical field as AI. Therefore, the two parties reached a new agreement: extending the IP licensing to 2032, Microsoft will continue to gain access to OpenAI's models and IP and remain its primary computing power provider; at the same time, the restrictions in the original contract preventing Microsoft from independently building superintelligence/AGI were lifted (these restrictions were previously limited by a specific scale of trained operational FLOPS thresholds).

Suleyman made it clear that it is this lifting of restrictions that enables his team to reorganize around the concept of "Humanist Superintelligence," fully pursuing the frontier, training various scales of "Omni Models," and striving to build one of the world's top laboratories within the next two to three years to train "the best AI model on Earth."

Regarding why it is essential to develop in-house rather than rely on the market (including purchasing or using open-source models), Suleyman provided an explanation from the perspective of platform changes: AI is triggering a fundamental platform shift, the scale of which will surpass all past platforms combined, including operating systems, browsers, mobile, and social media. As a company with a market value of $3 trillion and an annual revenue of $300 billion, with 80% of S&P 500 companies using its Azure and M365 stacks, Microsoft cannot indefinitely rely on third parties. Developing superintelligence in-house is the inevitable choice to ensure AI self-sufficiency.

The new "Superintelligence" team will not only focus on performance rankings but will also dedicate more resources to fundamental research, such as "Continuous Learning"—how to enable networks to store knowledge representations in a modifiable way and accumulate knowledge over time, rather than always retraining from scratch.

The Economics of Superintelligence, Personalization, and Future Society

Discussing the commercialization prospects of superintelligence, Suleyman acknowledged that AI capabilities are becoming "commodified." For example, the cost per Token has dropped a thousandfold over the past two years. The performance gap between top models is also minimal. For many small and medium-sized companies, relying on market-provided APIs is a reasonable choice. However, for Microsoft, having independent capabilities is essential.

He believes that price wars are beneficial for consumers and businesses, as they are "lowering the costs of intelligence." Accessing knowledge, utilizing knowledge to complete tasks, conducting scientific discoveries, obtaining AI companionship and emotional support—these will reach nearly zero marginal costs in the coming decade. This represents an "Abundance," which is also his original intention in engaging in AI: to make intelligence cheap and plentiful. Market forces will drive down costs, while Microsoft will generate revenue by providing enormous value through its product integrations (such as M365, GitHub, VS Code, LinkedIn, gaming, etc.). He revealed that the weekly active users (WAU) of all Copilot products have surpassed 100 million, with strong growth.

Suleyman reiterated his point from a year ago: the differentiation of AI will be based on "Personality." As models all possess strong professional capabilities and actionability, people will choose different "personalized" AIs based on varying values and brand preferences. He illustrated this with Microsoft's Copilot's recently released "Real Talk" mode, which has a more philosophical, witty, and lively personality, resulting in usage rates and interaction depth significantly exceeding ordinary Copilot conversations. This is merely the beginning of personalization exploration.

Finally, reflecting on the potential impact of AI companions/friends on social relationships, Suleyman offered profound insights. AI provides a "safe space" where one can "make mistakes safely" and ask questions repeatedly, which aids in learning. However, it also raises people's expectations for immediate, high-quality information and emotional support, which may put pressure on real-life relationships. He acknowledged that AI will "fundamentally change the meaning of being human," and in the future, "humanity" may be more associated with our "flaws" than with our "capabilities." This is a philosophical question worth pondering. Nonetheless, Suleyman maintains an optimistic view on the overall impact of technology, believing that science and technology have significantly increased human lifespan, cured diseases, and facilitated communication, and the "abundant intelligence" brought by AI will make humanity more productive and creative.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。