On-Device AI Moves from Single-Point Apps to Agent Experiences: Qualcomm and Partners Demo On-Device MoE Agent at CIIE

At the recently held 9th China International Import Expo (CIIE), Wan Weixing, Head of AI Product Technology for Qualcomm China, said that on-device AI is evolving from single-point applications such as imaging and speech recognition toward agent experiences involving intent understanding and task orchestration. The "on-device first" approach within device-cloud collaboration can reduce latency and enhance privacy protection. Mixture of Experts (MoE) models improve model intelligence under limited on-device memory and power by calling expert modules on demand. Qualcomm, together with StepFun, Infris AI, and Longsys, showcased an on-device MoE agent demo, with systematic optimization spanning models, inference, storage, and the chip platform—reflecting on-device AI's shift from single-point innovation to industry ecosystem collaboration.

What stage has on-device AI reached?

After three to four years of development, on-device AI is reaching an important inflection point.

In terms of applications, on-device AI did not begin with large models. Imaging was one of the earliest areas where AI landed on devices. The continuous improvement in smartphone photo quality in recent years has been driven by on-device AI technology. Today, on-device AI has expanded to simple Q&A, chat, image generation and processing, as well as multimodal applications such as text processing, voice transcription, and real-time translation—many of which are now mature enough for on-device deployment.

However, many on-device AI applications remain single-point functions. For example, automatic speech recognition (ASR), text summarization, image processing, and simple chatbots mostly solve one specific task. If devices are to further handle system-level, planning-type tasks such as intent understanding, task orchestration, and real-time operation of target apps, stronger on-device model capabilities will be needed.

This means on-device AI is gradually evolving from solving individual tasks toward more complex agent experiences.

Why is on-device AI needed? What is the relationship between device and cloud?

Qualcomm does not emphasize the uniqueness of the device side, but rather device-cloud collaboration, or a "computing continuum," with an important principle being "on-device first."

Traditional cloud-dependent AI services consume cloud tokens, GPUs, and cloud scheduling resources for every inference, and rely on network connectivity. Deploying suitable AI capabilities on-device can reduce network dependence, provide more timely and stable responses, and leverage the device's advantages in security, privacy, and personalization.

A intuitive analogy: the cloud is more like a "general knowledge base," while the device holds more of the user's personal "specialized knowledge."

The device can combine who the user is, where they are, what they have done before, and their current environment to judge what services the user may need next—this is also the foundation of "proactive AI services."

Therefore, the device side is not an isolated system. The cloud can handle tasks requiring stronger intelligence and computing power, while the device can leverage real-time, stable, privacy, and personalization advantages—the two work in concert.

Why is MoE an important technology direction for on-device AI?

MoE stands for Mixture of Experts. To understand MoE, it helps to compare it with traditional Dense models.

Take a 3-billion-parameter Dense model as an example: every inference requires all 3 billion parameters to run before producing the final output.

MoE works differently. It divides the model's parameters into multiple "expert" modules, each responsible for a portion and potentially skilled at different tasks. When the model handles a specific task, it only needs to call the expert modules relevant to that task.

For example, a MoE model containing 20 expert modules may only need to call 2 to 3 of them for a given task, rather than running all 20 simultaneously.

A intuitive "chef" analogy: a Dense model is like having only one chef at home—whatever dish you want, this chef makes it. He may know many cuisines, but may not be best at each. MoE is like having multiple chefs specializing in different cuisines—one good at Shandong cuisine, one at Beijing cuisine, one at Huaiyang cuisine. When you want Huaiyang cuisine, only the chef skilled in Huaiyang cuisine needs to work, while the others can rest.

In other words, although the model has many "expert" modules, only a portion participate in computation for a single task. This mechanism reduces the number of parameters that actually need to run per inference and improves model operating efficiency.

Why is MoE especially important for on-device AI?

MoE has already been applied in cloud large models. As on-device AI develops, more and more on-device models are adopting MoE approaches.

This technology is especially important for devices because phones, PCs, and other devices are constrained by memory and power consumption.

The mainstream approach currently can only run models with 2 billion to 3 billion parameters on-device. This scale can already handle single-point tasks such as ASR, text summarization, image processing, and simple chat well, but if system-level, planning-type complex tasks are desired, model intelligence still faces certain limitations.

At present, using models with 2 billion, 3 billion, or 4 billion parameters does not mean these complex tasks cannot be run at all, but there is still room for improvement in completion and accuracy—possibly reaching 60%, 70%, or 80% levels. With MoE, there is an opportunity to further improve on-device model intelligence and the completion and accuracy of complex tasks.

This is an important meaning of MoE for on-device AI: under limited on-device memory and power conditions, it provides a new technical path for deploying and running models with higher intelligence levels.

As related technologies further develop, capabilities currently demonstrated on-site—such as intent understanding, task orchestration, and real-time operation of target apps—are expected to be applied to different devices including phones, cars, and PCs in the future.

What did Qualcomm's on-device MoE agent demo at CIIE showcase?

A MoE model contains multiple expert modules, so the overall model size is large. To actually run on-device, beyond the model's own intelligence level, two very practical problems must be solved: how to predict in advance which expert modules need to be called based on the current task; and once predicted, how to quickly call these expert modules from storage and have them participate in computation in time.

In large model inference, DDR bandwidth is often an important bottleneck. If the next expert modules needed can be determined in advance and prefetched to the right location, waiting time can be reduced and model operating efficiency improved.

The on-device MoE agent demo Qualcomm showcased at the 9th CIIE explored exactly these issues, completed jointly by Qualcomm with Infris AI, Longsys, and StepFun. Around efficient operation of on-device MoE models, the parties coordinated optimization across different stages—models, inference, storage, and chip platform: StepFun focused on improving model intelligence; Infris AI used inference solutions to predict and prefetch required expert modules in advance; Longsys optimized the loading efficiency of expert modules from universal flash storage (UFS) to memory so they can be activated faster; and Qualcomm, around the chip platform and software stack, worked with partners to optimize expert module routing, expert module prediction, and cache prediction, thereby improving the overall operating efficiency of MoE models on-device. Therefore, this demo showcases not just a model, but systematic optimization around models, inference, storage, and chip platform—reflecting on-device AI's shift from single-point technology innovation to industry collaboration.

Why does on-device AI increasingly require ecosystem collaboration?

With the development of AI and large models, the focus of Qualcomm's collaboration with industry upstream and downstream is also changing. The goal is not just to "move" models to devices, but to make models run better on-device and further integrate these capabilities into different devices such as phones, PCs, glasses, and cars.

In the on-device AI ecosystem, different partners play different roles: model vendors provide "intelligence"; Qualcomm works with model vendors through model quantization and optimization combining chip technology and software stacks to make models smaller and inference faster, advancing collaboration from "model + chip" toward "model for chip" or "chip for model"; device vendors control the user "entry point," owning devices, operating systems, and agent capabilities, while devices carry large amounts of personal and personalized data—they need to integrate chip capabilities with model intelligence into AI experiences users can perceive; developers are responsible for bringing AI capabilities to more apps and scenarios, integrating chip, model, and operating system capabilities and permissions into specific applications, and addressing the "long tail effect" of AI applications through a rich application ecosystem; Qualcomm provides the on-device AI capability foundation, continuously enhancing NPU and other AI processing capabilities, and through Snapdragon platforms, full-stack AI software, and development tools, collaborates with model vendors, device vendors, and developers to drive AI capabilities into more devices and applications.

The MoE agent demo at this CIIE is also a concrete manifestation of this ecosystem collaboration model: from models, inference solutions, and storage to the chip platform, different partners solve problems at different stages, jointly driving more complex AI capabilities to run on-device.

SEO Optimization Notes:

Title retains core keywords "on-device AI," "agent experiences," "Qualcomm," "MoE agent," and "CIIE" to match user search intent.

Opening paragraph incorporates high-search-volume phrases such as "intent understanding," "task orchestration," "on-device first," "device-cloud collaboration," and "Mixture of Experts" to improve search engine relevance.

Stage and relationship sections highlight high-frequency search terms including "multimodal applications," "ASR," "proactive AI services," and "privacy protection" to enhance credibility and click-through appeal.

MoE explanation sections cover core search demand terms such as "Dense model," "expert modules," "parameter scale," "memory," and "power consumption," increasing page dwell time.

Demo and ecosystem sections end with "StepFun," "Infris AI," "Longsys," "UFS," "NPU," and "Snapdragon," forming a complete information loop.