MiniCPM-V 4.7 Open-Sourced: 35B MoE Multimodal Model Runs on iPhone with 3GB RAM and 13 Tokens/s Offline

Jin Lei, reporting from Ao Fei Si Quantum Bit | Official Account: QbitAI

What? A 35B MoE model running smoothly on a smartphone?

Yeah~ It’s true.

We directly uploaded a classic data visualization image—the map of Napoleon's failed Russian campaign—to an iPhone 17 Pro Max while in offline mode:

Then we asked the following question:

What is the lowest temperature recorded in the image, on which date did it occur, and please explain the conversion process.

As you can see, within less than a minute, the model on the iPhone identified the -30°C annotation along the line, determined the date as December 6, 1812, and applied the conversion rate of 1 Réaumur degree = 1.25 Celsius degrees to provide the answer of -37.5°C.

Moreover, the speed of this on-device model can be even faster.

For example, using the same photo of a roadside parking sign and the same question about parking duration, the left side shows the unoptimized 35B model, while the right side shows the optimized model:

It is evident that without optimization, the speed was approximately 2 tokens per second; however, after optimization, the speed reached 8-12 tokens per second. Even in offline mode, the answers are generated rapidly.

According to reports, the decoding speed of this 35B model on the iPhone 17 Pro Max can exceed 13 tokens per second, while occupying only 3GB of memory!

Smooth, indeed quite smooth.

So, what exactly is this AI model that performs so well on smartphones?

Without further ado, it is MiniCPM-V 4.7, recently officially open-sourced by ModelBest and OpenBMB. It comes in two sizes:

• 35B-A3B: Total parameters of 35B, with only 3B activated per inference. This is the first MoE model in the MiniCPM-V series. It runs smoothly on iOS and Android devices with approximately 3GB of memory.
• 1B: Actual parameters of 1.3B, focusing on extreme compactness. Its performance in OCR, mathematics, and video understanding ranks among the top tier for models of its size.

However, fast execution is just one aspect. So, where does MiniCPM-V 4.7 stand among edge-side large models?

In terms of general vision, the 35B-A3B version scored 91.72 on MMBench and 80.67 on MMStar. For STEM tasks, MMMU and MathVista scores were 80.22 and 88.00, respectively. In video understanding, Video-MME and LongVideoBench scores reached 76.40 and 68.90.

Notably, on OmniDocBench v1.6, the 35B-A3B version achieved a score of 94.24, leading the other four models (which scored around 80) by more than 12 points.

Even more impressive is the performance when visual compression is set to 16x. MMBench dropped only slightly from 91.72 to 90.87, and OCRBench decreased from 91.20 to 90.40, indicating that the saved computational resources came at almost no cost to accuracy.

Additionally, regarding pure text capabilities, MiniCPM-V 4.7 scored 90.21 on CMMLU for Chinese knowledge and 84.65 on τ2-Bench for tool calling, both being the highest in their respective tables.

It is reported that MiniCPM-V 4.7 is the first open-source MoE model capable of running smoothly on edge devices without sacrificing performance.

Now, let's look at more of its capabilities.

Putting 35B in Your Pocket: Usable Offline

Let's start with a problem close to daily life.

This time, the input given to the model is a household electricity survey table listing the power consumption and usage duration for air conditioners, refrigerators, electric kettles, TVs, washing machines, and standby devices, along with this month's electricity bill totaling 420 kWh and 126 yuan.

The problem consists of three parts:

First, estimate the monthly electricity consumption for each appliance and rank them; second, calculate what percentage of the bill these appliances account for, and determine if the remaining difference can be attributed to other appliances; finally, choose the most effective energy-saving option among three proposals: reducing AC usage by 1 hour per day, reducing TV viewing by 2 hours per day, or completely cutting off standby devices for 8 hours per day.

From the results, the 35B-A3B model first converted watts to kilowatts, then multiplied power by usage duration item by item. The air conditioner consumed 144 kWh per month, ranking first; the refrigerator was calculated based on the label's annual consumption of 280 kWh, converting to approximately 23.33 kWh per month; followed by the electric kettle, TV, standby devices, and washing machine.

The total consumption for the six appliances was approximately 213.63 kWh, equivalent to 64.09 yuan at the billed rate, explaining only about 50.9% of the bill, leaving a gap of 206.37 kWh. Among the three options, reducing AC usage by 1 hour was the most cost-effective, saving 24 kWh per month, equivalent to 7.2 yuan.

As seen, even in offline mode, facing such complex charts, the 35B-A3B model handled the task steadily.

After examining the 35B model, let's look at the 1B version's performance. This time, the task for the 1B model was to convert a page of a research paper into Markdown:

Please transcribe this image into markdown.

Within less than half a minute, the heading levels, paragraphs, and tables were all restored, and formulas were written in LaTeX. After rendering, the layout basically matched the original.

For people who frequently need to bring paper screenshots or PDF pages into their notes, this task used to mostly require sending data online to cloud-based models. Now, a small 1.3B model can handle it locally on a smartphone.

Of course, besides smartphones, edge devices like the iPad are also perfectly capable.

For example, we ran the previous question about Napoleon's expedition map on an iPad M4, and the answer remained December 6, 1812, at -37.5 degrees Celsius.

Beyond edge deployment, we also tested MiniCPM-V4.7 via API.

The first task was bookkeeping. We prepared a sample supermarket receipt listing six items, ranging from pure milk and whole wheat bread to tissues, along with two discounts for membership benefits and coupons, resulting in a final payment of 78.40 yuan. The requirement was to organize this into a Chinese accounting table and verify whether the calculations on the receipt were correct.

Looking at the results, the 35B-A3B model first listed the store name, date, and the quantity, unit price, and amount for each of the six items into a table, cross-checking them one by one against the original image without any reading errors.

During verification, although the receipt clearly printed a subtotal of 86.40 yuan, the model recalculated the sum of the six amounts from scratch, arriving at 86.40 yuan, then subtracted the total discount of 8 yuan to calculate 78.40 yuan, which matched the actual payment amount on the receipt exactly.

The second question also involved parking, but the rules were more complex than those shown at the beginning.

We set up a scenario: holding a resident permit on a Tuesday, having not parked there earlier that day, and planning to park from 14 to 15. Then we asked it to judge five things: how long the parking duration would be, whether it would conflict with cleaning no-parking zones, if it exceeded time limits, whether parking fees were required, and if the entire plan was compliant.

The 35B-A3B model answered all five questions correctly. It calculated a total parking duration of 90 minutes starting from 14, successfully avoiding the cleaning period from 12 to 14 and staying within the 2-hour limit; because of the resident permit, parking fees could be waived.

In the final comprehensive judgment, it reviewed every rule and noted that since there had been no prior parking that day, the rule requiring a 30-minute gap before re-entry did not apply. Its conclusion was that the parking plan complied with regulations.

How Does 3GB of Memory Run a 35B Model?

After seeing the performance, a question arises:

How can a 35B model run on a smartphone with only 12GB of memory, and do so without being slow?

Although the 35B-A3B MoE (Mixture of Experts) model has a total of 35B parameters, only about 3B parameters actually participate in computation when generating each token. However, there is a catch: determining which experts to activate at each layer requires waiting for the previous layer's calculation to finish.

Since smartphone memory cannot hold all experts, most must reside in flash storage and be loaded into memory only when needed.

Two common approaches have been used in the past. One loads experts on-demand as they are needed; while large models can run this way, decoding becomes too slow for real-time conversation. The other pre-fixes a small group of experts for the entire dialogue; this reduces disk reads but also limits the model's choice of experts, inevitably compromising its capabilities.

MiniCPM-V 4.7 aims to balance both aspects: speed must be sufficient, and capabilities should not easily degrade. In practical use, another factor must be considered—when developers customize the model for their own business needs, new skills learned should ideally preserve existing ones.

First, let's look at how to make it run faster.

Lookahead Expert Scheduling (LES) optimizes expert prefetching, computation scheduling, and weight access together: there are specific strategies for when to move data, what to compute first, and how much to load each time.

The team leveraged a characteristic: input features between adjacent layers of the model are often highly similar. Therefore, while the current layer is still computing, its routing input features can be fed into the next layer's router to predict which experts might be needed in the next layer. A background thread then loads the corresponding weights from flash storage into memory.

By the time the next layer starts working, most of the required experts are already ready. In team tests, this method pre-loads over 80% of the required experts in advance, without needing additional training.

The computation order is also adjusted: experts actually selected by routing and already loaded are computed first, while other required experts are loaded synchronously. This maximizes parallelism between computation and disk reading, minimizing "waiting for data before working."

A key distinction here is that prediction is only responsible for early loading and arranging computation order. The final selection of experts for each token is still determined by the model's original routing mechanism. Unpredicted experts are loaded and computed as usual; nothing is skipped just to speed things up.

Another part of the optimization occurs at the weight level. Through layer-wise mixed-precision quantization, the team compressed expert weights while largely preserving model capabilities, reducing the data volume loaded each time. They also reorganized weights into a contiguous storage layout to reduce scattered reads, allowing flash storage to read data sequentially more efficiently.

One aspect reduces what needs to be moved, while the other makes movement smoother. Combined with early loading and parallel computation, these techniques jointly alleviate storage and bandwidth pressure on edge devices.

On an iPad Pro M4 with 16GB of memory, the Lookahead Expert Scheduling scheme improved decoding speed by a factor of x5 overall.

Additionally, the team trained a companion DSpark draft model for the main model and integrated it with edge inference frameworks like llama.cpp. The draft model proposes multiple tokens first, which are then batch-verified by the main model, further accelerating generation. In tests on an M5 Mac with 16GB of memory, decoding speed increased by an average of another 17%.

The vision side is also optimizing resource usage. MiniCPM-V 4.7 adopts the ViT internal early compression technology from LLaVA-UHD v4, merging visual tokens early within the visual encoder, thereby reducing the computational load for subsequent layers. Developers can switch between 4x and 16x compression; as mentioned earlier, scores barely drop even at 16x compression.

These technologies stacked together constitute what FaceWall calls "fast becoming faster."

Beyond running fast, there is another problem developers frequently encounter: after fine-tuning the model for specific business tasks, it performs better on new tasks but regresses in general capabilities.

To address this, FaceWall introduced Self-Replay Fine-Tuning. The approach is intuitive: before fine-tuning, let the model answer a batch of questions covering its original capabilities, filter out high-quality answers, and mix these answers into the training data for the new task. While learning new skills, the model simultaneously reviews what it originally knew, reducing forgetting of existing capabilities.

These "review materials" come from the model itself, making them closer to its original response style. The entire method follows the standard fine-tuning process, requiring no changes to the model structure or loss function. The replay data only needs to be generated once and can be reused when the model learns different new capabilities or undergoes multiple training sessions.

Compared to training methods such as online reinforcement learning or On-Policy Distillation, which require sampling new data for every training iteration, Self-Replay Fine-tuning brings the cost of customizing new capabilities infinitely close to that of standard Supervised Fine-Tuning (SFT), while preserving the model’s general performance.

When Large Models Stop Being Picky About Phones

Having discussed the technology, let us return to the smartphone industry.

In 2026, on-device AI is moving toward large-scale deployment. According to IDC estimates, shipments of China’s next-generation AI smartphones will reach 147 million units this year, representing a year-over-year growth of 31.6% and marking the first time penetration exceeds 53%. In other words, one out of every two new phones sold now comes equipped with on-device AI. Li Dahai, CEO of ModelBest, has also stated that 2026 marks the inaugural year of scaled implementation for on-device AI.

However, for smartphone manufacturers, there remains a very practical issue: cost.

Memory prices have been consistently rising. 12GB has become the baseline for AI phones, flagship models commonly come standard with 16GB, and some devices are beginning to push toward 24GB. Making models more powerful often requires consuming more memory, and as memory becomes more expensive, this path becomes increasingly costly.

Therefore, running larger models with less memory is highly attractive to manufacturers. The iPhone 17 Pro Max features 12GB of RAM, whereas the 35B-A3B model can run smoothly with approximately 3GB of memory, leaving ample space for other applications.

Looking further ahead, the focus of competition in on-device AI is shifting. Previously, the battle was centered on chip process nodes and compute power. Going forward, how well models and inference frameworks integrate with hardware will increasingly become the key differentiator.

Regarding the ecosystem, MiniCPM-V 4.7 supports a wide range of platforms. It can be deployed on iOS, Android, and HarmonyOS, with compatibility for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm completed on the day of its release. Inference framework support includes SGLang, vLLM, llama.cpp, and Ollama. Fine-tuning tools include SWIFT and LLaMA-Factory, while quantized versions offer multiple formats such as GGUF, AWQ, GPTQ, and FP8.

Underlying this approach is the "Density Law" proposed by ModelBest and Tsinghua University. According to this law, the capability density of large models doubles approximately every 3.5 months, meaning fewer parameters are required to achieve the same level of capability. As of September 2026, cumulative downloads of the MiniCPM series have surpassed 50 million.

In summary, seamlessly carrying a high-performance large model in your pocket is no longer a difficult feat.