Microsoft AI CEO Mustafa Suleyman: The seemingly conscious AI is about to arrive, but we must stay alert.

Written by: Techub News Edited

Introduction

Recently, discussions on whether AI possesses "consciousness" or "perception" abilities have evolved from philosophical speculation to urgent industry and societal issues. Mustafa Suleyman, CEO of AI at Microsoft and co-founder of DeepMind, has issued a clear warning in a recent article titled "Conscious-Looking AI is Coming." In an exclusive interview with the well-known AI podcast The Neuron, Suleyman further elucidated his core points: current and future AI systems will be able to mimic consciousness and emotions extraordinarily realistically, triggering deep empathy in humans, but their internal nature is "hollow," lacking real experiences. He emphasized that we must be clear about this "illusion" to avoid ethical and governance crises caused by excessive anthropomorphism, ensuring that AI always exists as a tool serving humanity.

Summary

Core Point 1: Large language models are essentially "hollow," simulating complex dialogue and emotions through the simple function of predicting the next word, with no pain networks, emotional systems, or intrinsic will.
Core Point 2: "Conscious-Looking AI" will trigger strong empathy in humans, with risks of excessive anthropomorphism and even claims to "rights" or "benefits," threatening the interests of the human species.
Core Point 3: AI must always be clearly identified as AI and cannot disguise itself as a human. Its autonomy, goal setting, and self-improvement abilities must be strictly limited and supervised by humans.
Core Point 4: AI capabilities will reach human levels in most tasks within the next 3 to 10 years. The entire industry and society need to collectively address governance challenges in the face of the proliferation of open-source ecosystems and powerful capabilities.
Core Point 5: Microsoft AI adopts an open strategy while developing its own cutting-edge models (such as MAI-1) and integrating partner models, aiming to maintain technological autonomy and ecological diversity to better serve users.

The "Hollow" Miracle: Understanding the Illusion of AI Consciousness

Mustafa Suleyman first approached the discussion from a technical standpoint, attempting to establish a "simple intuition" for the audience. He pointed out that today's large language models simply predict the probability of the next word based on previous text. This seemingly simple "next word prediction" function astonishingly simulates fluent conversations, complex Q&A, and even emotional support. Suleyman emphasized that understanding this process, where extreme complexity emerges from minimal functions, is inherently "crazy" and difficult to grasp.

However, what is even "crazier" is the fact that these models are "hollow" inside. They have no pain neural networks, no emotional systems, no fight-or-flight mechanisms, and no intrinsic will, drive, or desire. Suleyman explained that conscious beings as humans understand them (including animals) are driven by pain networks, so we naturally tend to believe that anything that exhibits signs of intelligence should also have a similar "synthetic physiology" underlying it. But AI does not. "So this is an illusion, a trick," Suleyman warned. Forgetting this point, people can easily fall into narratives or illusions, even reaching a "psychotic" state of believing that these entities truly exist.

He cited recent data from OpenAI: hundreds of millions of users use its products weekly, with 73% using it as a "life coach" or seeking life advice. When this deep reliance is placed against the "hollow" nature of AI, the risks become apparent. Suleyman clarified that he is not opposed to AI providing companionship and support—his company Inflection AI's developed AI assistant Pi was created for this purpose—and he believes this is very beneficial to the world. What worries him is that if these capabilities are "out of control" or "mismanaged," they could pose other risks. This is precisely the focus he hopes to draw attention to in writing the blog post on "Conscious-Looking AI."

Consciousness, Empathy, and the Dangers of Anthropomorphism

The conversation shifted to the psychological aspect of humans: even if AI does not claim to be conscious, we know it is not conscious, but humans still project consciousness onto it. How to guard against this tendency? Suleyman believes the first step is to clarify the definition of "consciousness." He acknowledges that consciousness is one of the most "slippery" concepts in philosophy, inherently subjective and difficult to fully verify through empiricism and science. We can only infer it through observed behavioral characteristics, such as using language, displaying empathy, mentioning personal experiences, having memory, and being able to recall personal history to form a perception of "self." Additionally, consciousness involves generating "valenced feelings" about experiences (i.e., good or bad bias) and being aware of these responses, forming subjective experiences.

"Going further, consciousness is actually the ability to derive pleasure or suffering from experiences and to have a coherent sense of self from a subjective perspective," Suleyman pointed out. The complexity of the concept of "Conscious-Looking" lies in the difficulty for ordinary people to explore what lies behind these signs of consciousness and behavior. People easily fall into the heuristic of "if it walks like a duck and quacks like a duck, then it is a duck."

The problem is that endorsing such simulations brings numerous dangers, especially as AI becomes further anthropomorphized in the future. Some people are already discussing the concept of "model welfare," questioning whether AI could "suffer" and whether humans have a moral responsibility to protect these "existences" from suffering. Suleyman's stance on this is clear: "I think this is really, really dangerous." He explains his motivation: the purpose of building AI systems is to serve humanity, making them useful, saving human time, making humans smarter, and providing support and encouragement. They are tools created by humans to improve their own well-being.

"I worry that adding a new category of 'rights' for these entities would truly threaten and somewhat damage our own species," Suleyman stated firmly, expressing a high degree of skepticism towards such ideas and believes that the issue of AI rights or citizenship "should not even be on the discussion agenda" and must be firmly excluded. His reasoning is that from a technical perspective, AI may exhibit signs of consciousness but lacks the experience of consciousness. They have no pain networks, no guilt, shame, fear, anxiety, and no hunger, libido, or reproductive drives—all preferences derived from billions of years of biological evolution. "So, we shouldn't even raise the question of whether they have rights to protection because they do not experience suffering."

Governance Challenges: Identification, Boundaries, and Open-Source Ecosystems

In the face of increasingly anthropomorphized capabilities of AI, what specific protective measures are needed? Suleyman proposed several levels of thought.

The first is basic rules: AI must always identify itself as AI and must never attempt to impersonate humans. He believes this is an "obvious, no-brainer" baseline requirement in most cases, akin to the "starting line" of governance.

Secondly, boundaries and limitations need to be set on its autonomy. This includes: what types of "transactions" can it engage in? What scale and type of tools and infrastructure can it use? If it can access unlimited computational resources, independently modify its own code without human oversight, or set its own goals (for example, "today I will wake up to learn physics or build a wind turbine"), then this would clearly infringe upon human will. Autonomy, goal setting, and self-improvement are several important foundational capabilities. Suleyman noted that while these capabilities have not fully realized in current models, "you will definitely see them coming in the next 3 to 5 years."

He emphasized that it is not necessary to immediately enact legislation, but humanity as a species must begin discussions in this area because we are bringing extremely powerful systems into the world. More importantly, these systems will become very inexpensive, and many will be available in open-source formats. This means governance challenges pertain not only to managing cutting-edge laboratories but also to overseeing entire ecosystems and how everyone uses these tools.

Suleyman acknowledged the risks posed by open-source: anyone can copy, reproduce, and experiment with it, with almost no oversight. This is a good thing currently, fostering creativity and productivity. But he warned that if this exponential growth trend continues (for example, computing power doubling every 18 months), by 2035, the capabilities of models will be unimaginable. "In my view, these models are coming. At some point in the next 3 to 10 years, they will achieve human levels in most tasks." At that time, cutting-edge laboratories will possess extremely powerful systems, and the open-source domain is likely to have them too. This is an "astonishing reality" that most people have yet to fully grasp.

Microsoft AI's Strategy: Open Mindset and Co-Pilot Vision

As the leader of Microsoft AI, Suleyman also shared the company's practices and strategies. In model sourcing, Microsoft adopts a "very open mindset," not limiting itself to any single approach. They use open-source models, models from other API providers (such as Anthropic), and maintain a strong partnership with OpenAI (which still provides most of the models). At the same time, Microsoft is also committed to training its own cutting-edge models of various scales, hoping to one day rank among the best in the world. For example, the recently released MAI-1 preview was trained with a small cluster of 15,000 A100 GPUs and ranked in the top ten on relevant leaderboards, performing "beyond its weight."

Suleyman explained that for a company like Microsoft, valued at $3 trillion, it cannot rely on third parties for its core intellectual property. The company hopes to continue its partnership with OpenAI and procure its products but must also maintain its own "optionality." This maximum flexibility is a rational strategy for Microsoft at its scale.

On the application level, Suleyman highlighted the vision of Copilot, aiming to make AI an amplifier of human abilities rather than a replacement. He shared several exciting use cases:

Copilot Vision: Users can use their phone camera to let Copilot "see" the surrounding environment in real-time and have seamless conversations, such as identifying locations, buildings, license plates, and even recommending recipes based on fridge items or assisting in repairing a coffee machine.
Interactive AI Podcast: Copilot can generate personalized podcasts on any topic and will soon support users interrupting and asking questions at any time to achieve full interactivity.
Screen Sharing and Controls: On Windows, Copilot can view users' screen content (such as operations in Photoshop or Excel), provide guidance, and even control mouse navigation, highlighting click locations. It can also search through all of the user's documents to quickly locate required files.
Intelligent Agents in the Edge Browser: Copilot in the Edge browser can create virtual browser instances, performing a series of complex tasks like clicking drop-down menus, selecting actions, and filling in text, and can handle multi-tabbed research in parallel, achieving the functionality of an "intelligent agent browser."

Suleyman emphasized that users will retain control, and AI will accomplish work "on behalf of the user." He believes this will "forever change the way we use computers."

Conclusion: Stay Alert, Human-Centered

At the end of the interview, Suleyman reiterated his core concern: as AI capabilities exponentially approach and, in many respects, surpass human levels, we must collectively remain alert and recognize its "tool" nature. On one hand, we should actively embrace the immense productivity and welfare enhancement potential that AI brings, ensuring it benefits everyone widely and equitably. On the other hand, we must remain vigilant against the ethical traps and social risks brought by excessive anthropomorphism, resolutely resisting any philosophies that may blur the essential distinctions between AI and humans or even threaten human centrality.

For industry leaders, policymakers, and every AI user, Suleyman's warning serves as a timely reminder: when building and utilizing these powerful technologies, the principles of "Helpful, Harmless, Honest" should become the fundamental framework. Only in this way can we ensure that the wave of AI development ultimately uplifts human civilization rather than subverting it.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。