Article｜Li Zhaofeng

Editor｜Zhang Yuxin

Yesterday, Doubao Mobile Assistant released the consumer version. This AI assistant, based on Doubao App and developed in collaboration with mobile manufacturers at the operating system level, has its first landing model as Nubia NaviX Ultra, which will officially go on sale on September 16.

At the end of last year, the technical preview version of Doubao Mobile Assistant was implemented on the Nubia M153 engineering prototype, focusing on demonstrating how AI can "operate the phone" across applications. In less than a year, the newly launched consumer version emphasizes daily usability: in addition to screen Q&A, local memory, and retrieval, the "operate phone" feature is open in Beta form; Doubao also introduced SAEP (Screen Automation Operation Declaration Protocol) with a 30-day public notice period.

According to the protocol, third-party applications can independently declare whether to allow the AI assistant to perform screen automation, and can also restrict specific operations. During the 30-day public notice period, third-party applications that have not explicitly agreed to be accessed will not operate by default.The consumer version of Doubao Mobile Assistant further adds the choice for applications: third-party applications can refuse the entire GUI or individually restrict operations such as publishing, deleting, and claiming benefits.

Those engaged in GUI (Graphical User Interface) Agent research believe: "In the future, there must be a native AI assistant in phones. This may not necessarily be done by mobile manufacturers themselves, but it must ultimately be achieved at the system level." As the NaviX Ultra and a number of system-level mobile assistants enter the consumer market, system-level AI assistants are heading towards the daily usage scenarios of ordinary users.

Stepping out of the chat box, mobile AI begins to complete tasks across Apps

In July this year, Ni Fei, Senior Vice President of ZTE Corporation and President of Nubia, stated in a media interview: "What users really want is 'peace of mind,' but today's mobile phone functions are increasing and operations are becoming more complex. This contradiction has accumulated to a certain extent, requiring a brand-new product form to respond."

This is precisely the problem Doubao Mobile Assistant is trying to solve: it is embedded into the mobile phone system and can search current screen and local information after user authorization, call calendar and other system tools, or enter other Apps to complete operations within the Apps. Users can initiate tasks through the AI key, and multiple tasks can queue or temporarily jump the line.

For example, in the official demonstration of Doubao Mobile Assistant, after the user takes a photo of their new semester schedule, they can ask the assistant to add the schedule to their personal calendar; they can also ask it to hail a taxi to the address displayed on the screen, add Yindu Building as a stop, send the license plate number to a colleague on Feishu after getting in the car, and remind them to come down in 10 minutes.

This method of use already had a prototype in last year's technical preview version—the Nubia M153. In a public demonstration, the user asked Doubao to "pick a cheap similar item on Taobao" based on a photo of a blood pressure monitor in the album, and then "send the link to Zhang San." Doubao identified the product and recognized that "Zhang San" is a QQ friend, subsequently completing operations across photo album, e-commerce, and social applications.

It is reported that the M153 used the Android INJECT_EVENTS system-level permissions. After obtaining user authorization, even if third-party Apps do not provide interfaces for AI, Doubao can simulate taps, slides, and input; when encountering sensitive steps like passwords or facial recognition, the phone will be returned to the user.

However, in actual use, cross-App operations involving some login verification and platform risk control may interrupt tasks. Subsequently, Doubao proactively restricted high-risk scenarios such as financial payments and incentive grabbing.

In February this year, Google tested the Gemini multi-step tasks on the Galaxy S26 series and some Pixel 10 models: it can arrange a ride-hailing service for colleagues that includes multiple stops. After tasks transition to the background, users can still check progress through notifications, take over at any time, and confirm before purchase. Matthew McCullough, Vice President of Android Developer Product Management at Google, wrote in the official blog: "The standard for measuring the success of applications is shifting from 'making users open the App' to 'truly completing users' tasks.'

Since 2026, a batch of system-level mobile AI assistants has emerged.

In the HarmonyOS 7 Developer Beta released in June, Huawei Xiaoyi accessed over 200 system-level perception data, capable of identifying time, location, and precautions from interview invitations and writing them into memos; it can also ask Xiaoyi to find files on the computer and send them to the phone.

At the 2026 World Artificial Intelligence Conference (WAIC) in July, Nubia NaviX Ultra, STEPX Neo, and Honor Robot Phone made a concentrated appearance. The Amoo assistant equipped on STEPX Neo, upon receiving the instruction "help me handle tomorrow's report," will read the Feishu calendar and group messages, break down tasks, generate PPTs, and then send the documents back to Feishu; it will request permission before modifying documents, and involve payment will require user confirmation.

Almost simultaneously, the relationship between super Apps and mobile AI assistants is also changing.

In June this year, WeChat confirmed its collaboration with Huawei, Honor, Xiaomi, OPPO, and vivo on A2A (agent-to-agent communication) capabilities, allowing mobile assistants to submit messaging or audio-video call commands, to be executed by WeChat with results returned, using dual authorization throughout. In August, vivo's Lanxin Xiaov with Alipay service intelligent agent "Abao" collaborated, allowing users to say "help me call a taxi to the high-speed train station" or "help me issue a taxi invoice," and related services can be completed within the conversation.

It is evident that in 2026, mobile AI assistants have entered a period of intensive release. In addition to Nubia, which is equipped with Doubao Mobile Assistant, Google, Huawei, STEP, and Honor have successively brought Agents to the mobile operating system level, while super Apps like WeChat and Alipay have also begun to accept commands issued by system assistants.

How can AI operate your screen without overstepping boundaries?

As mobile AI assistants adopting the GUI route become more popular, they increasingly need to address three permission issues: whether users authorize, whether applications allow access, and who confirms operations prior to sending, ordering, and payment. From Doubao's introduction of the SAEP (Screen Automation Operation Declaration Protocol), we may glimpse at security measures regarding these permission boundaries.

Currently, CUA (Computer Operating Agent)has takenthe GUI technology routeas mainstream,solving the problem of how AI directly operates software and how to limit accessible scope.Some researchers describe CUA as "equipping your computer with a particularly versatile programmer who knows everything, can code, and also understands network security."

The current operational method is: for tasks that can be structured, the phone will prioritize calling codes, commands, and tools; only when encountering software without interfaces will it switch to recognizing screens and clicking buttons. GUI is more like a fallback capability, allowing the Agent to continue operating on software that has not yet opened up to AI—there are numerous mobile Apps with various temporary pages constantly changing, making it impossible to interface them all, requiring GUI to cover the unconnected parts.

A GUI researcher told Smart Emergence: "You cannot rely on all App developers to adapt for AI; GUI is definitely a necessary supplement." Another practitioner mentioned, "In the short term, many software in the market will not have AI capabilities, especially large-scale software, even if there are interfaces, they may be very complex, and cannot be accommodated at once."

Mobile Agents depend more on the application ecosystem than computer Agents. On computers, the operating system can directly decide which software can connect with the Agent; however, on mobile devices, the operating system lacks such authority, with different services encapsulated within their own Apps, and permissions for location, camera, address book, etc. are jointly controlled by the system and applications. If a third-party App neither opens an interface nor allows AI to operate the interface, the mobile assistant cannot continue performing tasks.

Currently, the GUI displays every step of the operation on the screen. One researcher stated, "Everyone can understand it and see what this AI assistant is doing." Users can then decide whether to pause or take over.

But "visibility" does not equal "doing well." The research team at Carnegie Mellon University this year launched the mobile Agent evaluation benchmark iOSWorld, simulating the digital life of the same user with 26 self-developed iOS applications and designing 133 tasks. The best-performing model had only a 37% success rate in cross-application tasks, and real consumption scenarios still face constantly changing pages, identity verification, and platform risk control.

Compared to App scenes, the mobile system level also aggregates user personal information such as albums, address books, locations, and calendars. The more complete the information the mobile AI assistant has, the more tasks it can accomplish. Therefore, the next competition will involve two capabilities: one is "how to generate more and better data," and the other is "how to organize steps and collaboration between various Agents."

However, more capabilities also mean higher requirements for the ecosystem.

In this regard, Doubao Mobile Assistant chose to launch SAEP(Screen Automation Operation Declaration Protocol), allowing third-party applications to declare whether to allow screen automation before the GUI Agent executes and which operations cannot be executed.

"SAEP is somewhat like the Robots protocol of the AI era (web crawler access rules). Websites use this protocol to tell search engines and other automated programs which pages are allowed to be crawled and which are not; SAEP allows applications to declare whether Agents can enter the application and which operations must stop," one researcher explained.

After joining SAEP, third-party applications can refuse all screen automation or only restrict specific operations such as content publishing, deletion, checking in for lotteries, and claiming benefits; refusal declarations take effect immediately. During the 30-day public notice period, except for system-built applications, applications under ByteDance, and third-party applications that have explicitly agreed to be accessed, other applications will not operate by default. After the public notice period ends, applications that have not expressed their stance will gradually open according to risk levels, while applications that have clearly refused will never be operated.

Applications express their intentions through SAEP, and the mobile system is responsible for reading declarations, restricting GUI permissions, and returning to the user during sensitive steps. This requires cooperation between model companies and terminal manufacturers.

Previously, Ni Fei, President of Nubia, summarized the collaboration between Doubao and Nubia as: "Model companies solve whether AI can do it, while terminal manufacturers solve whether users can really use it." According to the division of labor disclosed by Ni Fei, Doubao is responsible for large models andAgent capabilities, while Nubia is responsible for systems, hardware, product engineering, quality, channels, and user services. The cooperation scope covers models, operating systems, and complete machine engineering.

In this division, permissions and trust will ultimately be undertaken by the operating system, meaning that the relationship between the mobile operating system and the Agent will become increasingly integrated.

In August this year, at the Alipay AI ecosystem partner conference held in Hangzhou, Zhou Wei, Vice President of Vivo and Head of the AI Global Research Institute, stated: "Hardware forms will not disappear, but the operating system will be reconstructed—from managing Apps to managing Agents, managing permissions, managing trust."

Earlier, in July, the National Internet Information Office first announced the registration of seven mobile-side generative AI services, involving Apple Intelligence, Huawei Xiaoyi AI large model, OPPO AndesGPT large model, vivo Lanxin edge large model, Xiaomi Surge AI, Samsung Galaxy AI, and Nubia Doubao mobile large model. Mobile system-level AI assistants are entering a period of industry explosion.

A GUI researcher told Smart Emergence that, compared to independent AI applications, mobile phones have more stable user relationships: "If you only do AI software, users may use whoever provides it for free; but mobile phones are different, as a phone is often used for several years and will accompany users for a long time."

On September 16, the NaviX Ultra will officially go on sale. At that time, Doubao Mobile Assistant will officially enter the consumer market, and the SAEP will also undergo real user and application developer testing.

Image source｜Doubao Mobile Assistant

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。