ModelBest Open-Sources MiniCPM-V 4.7: 35B MoE Multimodal Model Runs Offline on iPhone with 3GB RAM

ModelBest Open-Sources MiniCPM-V 4.7

Jin Lei, reporting from Aofeisi

QbitAI | WeChat Official Account: QbitAI

What? A 35B MoE model running smoothly on a smartphone?

Yeah~ It’s true.

We directly uploaded a classic data visualization image—the map of Napoleon's failed Russian campaign—to an iPhone 17 Pro Max while in offline mode:

Then we asked the following question:

What is the lowest temperature recorded in the image, on which date did it occur, and please explain the conversion process.

As you can see, within less than a minute, the model on the iPhone identified the -30 degree annotation along that line, determined the date as December 6, 1812, and applied the conversion rate of 1 Réaumur degree equals 1.25 Celsius degrees to provide an answer of -37.5 Celsius degrees.

Moreover, the inference speed of this on-device model can be even faster.

For example, given the same photo of a roadside parking sign and the same question about parking duration, the left side shows the unoptimized 35B model, while the right side shows the optimized version:

It is evident that without optimization, the speed was approximately 2 tokens per second; however, after optimization, the speed reached 8-12 tokens per second. Even in offline mode, answers are generated rapidly.

Reportedly, this 35B model achieves a decoding speed of over 13 tokens per second on the iPhone 17 Pro Max, while consuming only 3GB of memory!

Smooth, indeed quite smooth.

So, what exactly is this AI model that performs so well on smartphones?

Without further ado, it is MiniCPM-V 4.7, recently officially open-sourced by ModelBest and OpenBMB. It comes in two sizes:

• 35B-A3B: Total parameters of 35B, with only 3B activated per inference. This is also the first MoE model in the MiniCPM-V series. It runs smoothly on iOS and Android devices requiring approximately 3GB of memory.
• 1B: Actual parameters of 1.3B, focusing on minimal footprint. Its performance in OCR, mathematics, and video understanding ranks among the top tier for models of similar size.

However, fast inference is just one aspect. How does MiniCPM-V 4.7 compare within the realm of edge-side large models?

In terms of general vision, the 35B-A3B variant scored 91.72 on MMBench and 80.67 on MMStar. For STEM questions, MMMU and MathVista scores were 80.22 and 88.00, respectively. In video understanding, Video-MME and LongVideoBench scores reached 76.40 and 68.90.

Notably, on OmniDocBench v1.6, the 35B-A3B achieved a score of 94.24, leading the other four models (which hovered around 80 points) by more than 12 points.

Even more impressive is its performance when visual compression is set to 16x. MMBench dropped only slightly from 91.72 to 90.87, and OCRBench fell from 91.20 to 90.40, indicating that the saved computational resources came at almost no cost to accuracy.

Additionally, regarding pure text capabilities, MiniCPM-V 4.7 scored 90.21 on CMMLU (Chinese knowledge) and 84.65 on τ2-Bench (tool calling), both being the highest in their respective tables.

It is reported that MiniCPM-V 4.7 is the first open-source MoE model capable of running smoothly on edge devices without sacrificing performance.

Next, let us examine more of its capabilities.

Putting a 35B Model in Your Pocket, Usable Offline

Let's start with a problem closely related to our daily lives.

This time, the model was given a household electricity usage survey listing the power ratings and usage durations for an air conditioner, refrigerator, electric kettle, television, washing machine, and standby devices. It also included this month’s electricity bill: 420 kWh costing ¥126.

The task consisted of three questions:

First, estimate each appliance’s monthly energy consumption and rank them; second, calculate what percentage of the bill these appliances account for and determine whether the remaining difference can be attributed to other devices; finally, identify the most effective option among three energy-saving strategies: reducing air conditioner usage by one hour per day, cutting TV viewing time by two hours per day, or completely powering off standby devices for eight hours daily.

In terms of results, 35B-A3B first converted watts to kilowatts, then multiplied power by usage duration item by item. The air conditioner consumed 144 kWh per month, ranking first; the refrigerator was calculated based on its label’s annual consumption of 280 kWh, translating to approximately 23.33 kWh per month; followed in order by the electric kettle, television, standby devices, and washing machine.

The six appliances totaled approximately 213.63 kWh, equivalent to ¥64.09 at the billed rate, explaining only about 50.9% of the total bill, leaving a gap of 206.37 kWh. Among the three options, reducing air conditioner usage by one hour per day proved most cost-effective, saving 24 kWh (¥7.2) per month.

As we can see, even when offline, 35B-A3B handled such complex charts with ease.

After reviewing 35B, let’s look at the performance of the 1B model. This time, the task assigned to 1B was converting a page from a research paper into Markdown:

Please transcribe this image into Markdown.

In less than half a minute, heading levels, paragraphs, and tables were accurately reconstructed, and formulas were rendered in LaTeX, matching the original layout almost perfectly.

For those who frequently need to transfer screenshots of papers or PDF pages into notes, this type of task previously required cloud-based models connected to the internet. Now, a small 1.3B model can handle it entirely locally on a smartphone.

Of course, besides smartphones, edge devices like iPad are equally capable.

For example, when we ran the earlier question about Napoleon’s expedition map on an iPad M4, the answer remained December 6, 1812, at -37.5°C.

Beyond edge deployment, we also tested MiniCPM-V 4.7 via API.

The first task involved bookkeeping. We provided a sample supermarket receipt listing six items—from pure milk and whole wheat bread to tissues—along with member discounts and coupon reductions, resulting in a final payment of ¥78.40. The requirement was to organize this into a Chinese accounting table and verify the calculations on the receipt.

From the results, 35B-A3B listed the store name, date, and quantities, unit prices, and amounts for all six items in a table, cross-checking against the original image without any reading errors.

During verification, although the receipt clearly stated a subtotal of ¥86.40, the model recalculated the sum of the six item amounts from scratch, arriving at ¥86.40, then subtracted the combined ¥8 discount to reach ¥78.40, exactly matching the actual payment amount on the receipt.

The second task also related to parking but involved more complex rules than our initial demonstration.

We set up a scenario where a resident permit holder parked on Tuesday, had not yet parked there that day, and planned to park from 14 to 15. The model was asked to evaluate five aspects: total parking duration, potential conflict with street sweeping restrictions, compliance with maximum stay limits, applicable fees, and overall plan validity.

35B-A3B answered all five questions correctly. It calculated a total parking duration of 90 minutes starting at 14, avoiding the 12–14 street sweeping window and staying within the two-hour limit. Since the driver held a resident permit, no parking fee was required.

In its final comprehensive assessment, the model reviewed every rule and noted that since no prior parking occurred that day, the “30-minute re-entry interval” restriction did not apply, concluding that the parking plan was compliant.

How Does 3GB RAM Run a 35B Model?

After seeing the results, a natural question arises:

How can a 35B model run smoothly on a smartphone with just 12GB of memory—and do so without significant latency?

Although 35B-A3B is a MoE model with 35 billion parameters, only about 3 billion are actively involved in computation during token generation. However, there’s a catch: determining which experts to activate at each layer requires waiting until the previous layer finishes processing.

Moreover, smartphone memory cannot accommodate all experts simultaneously; most must reside in flash storage and be loaded on demand.

Traditional approaches typically fall into two categories. One loads experts only as needed—enabling large models to run but causing decoding speeds too slow for real-time conversation. The other pre-selects a fixed subset of experts for entire conversations, reducing disk reads but limiting the model’s expert selection capacity and compromising performance.

MiniCPM-V 4.7 aims to balance both: maintaining sufficient speed while preserving capability. In practical use, another consideration emerges—when developers customize the model for specific business needs, newly acquired skills should not degrade existing ones.

Let’s first examine how speed improvements were achieved.

Lookahead Expert Scheduling (LES) optimizes expert prefetching, computational scheduling, and weight loading together: decisions about when to load, which computations to prioritize, and how much data to fetch each time are carefully engineered.

The team leveraged a key characteristic: input features across adjacent layers tend to be highly similar. Therefore, while the current layer is still computing, its routing input features can be fed into the next layer’s router to predict which experts will likely be needed. A background thread then loads corresponding weights from flash storage into memory.

By the time the next layer begins execution, most required experts are already prepared. Testing showed this method preloads over 80% of necessary experts without requiring additional training.

Computation order was also adjusted: experts actually selected by routing and already loaded are computed first, while remaining required experts are loaded concurrently. This maximizes overlap between computation and disk I/O, minimizing idle wait times.

A crucial distinction exists here: prediction solely guides prefetching and computation ordering. Final expert selection for each token remains determined by the model’s native routing mechanism. Unpredicted experts are still loaded and computed normally—no shortcuts are taken to boost speed.

Another part of the optimization occurs within the weights themselves. Through layer-wise mixed-precision quantization, the team compressed expert weights while largely preserving model capabilities, thereby reducing the data volume loaded during each inference step. They then reorganized the weights into a contiguous storage layout to minimize scattered reads, allowing flash memory to perform sequential reads more efficiently.

One strategy reduces the amount of data that needs to be moved, while the other makes the movement smoother. Combined with prefetching and parallel computation, these approaches collectively alleviate storage and bandwidth pressures on edge devices.

On an iPad Pro M4 with 16GB of RAM, the lookahead expert scheduling scheme achieved a comprehensive 5x improvement in decoding speed.

Additionally, the team trained a companion DSpark draft model for the main model and integrated it with edge-side inference frameworks such as llama.cpp. The draft model proposes multiple tokens first, which are then batch-verified by the main model, further accelerating generation. In tests on an M5 Mac with 16GB of RAM, this approach yielded an average additional 17% boost in decoding speed.

Optimizations are also being applied on the vision side. MiniCPM-V 4.7 adopts the early compression technique within the ViT from LLaVA-UHD v4, merging visual tokens inside the vision encoder at an early stage. This reduces the computational load for subsequent layers. Developers can switch between 4x and 16x compression ratios; as mentioned earlier, performance scores remain virtually unchanged even at 16x compression.

The combination of these technologies is what ModelBest refers to as "faster still."

Beyond speed, developers frequently encounter another challenge: after fine-tuning a model for specific business tasks, performance on new tasks improves, but general capabilities often degrade.

To address this, ModelBest introduced Self-Replay Fine-Tuning. The approach is intuitive: before fine-tuning, the model answers a set of questions covering its original capabilities. High-quality responses are selected and mixed into the training data for the new task. As the model learns new skills, it simultaneously reviews what it already knows, reducing catastrophic forgetting of existing capabilities.

These "review materials" originate from the model itself, making them closer to its native response style. The entire method follows the standard fine-tuning workflow, requiring no modifications to the model architecture or loss function. The replay data only needs to be generated once and can be reused when the model learns different new capabilities or undergoes multiple training rounds.

Compared to online reinforcement learning or On-Policy Distillation, which require sampling new data for every training iteration, Self-Replay Fine-Tuning brings the cost of customizing new capabilities infinitely close to that of standard Supervised Fine-Tuning (SFT), while preserving the model’s general performance.

When Large Models No Longer Pick Phones

Having discussed the technology, let us return to the smartphone industry.

In 2026, edge AI is moving toward large-scale deployment. According to IDC projections, shipments of China’s next-generation AI smartphones will reach 147 million units this year, representing a 31.6% year-over-year increase, with penetration rates breaking the 53% threshold for the first time. In other words, one out of every two new phones sold now comes equipped with edge AI. Li Dahai, CEO of ModelBest, has stated that 2026 marks the inaugural year of scaled implementation for edge AI.

However, for smartphone manufacturers, there remains a very real issue: cost.

Memory prices continue to rise. 12GB has become the baseline for AI phones, flagship models generally come standard with 16GB, and some devices are beginning to push toward 24GB. Making models stronger often requires more memory, and as memory becomes more expensive, this path becomes increasingly costly.

Therefore, running larger models with less memory is highly attractive to manufacturers. The iPhone 17 Pro Max features 12GB of RAM, whereas the 35B-A3B model runs smoothly with approximately 3GB of memory usage, leaving ample space for other applications.

Looking further ahead, the focus of competition in edge AI is shifting. Previously, the battle was over chip process nodes and compute power. Going forward, how well models and inference frameworks integrate with hardware will increasingly become the key differentiator.

Regarding the ecosystem, MiniCPM-V 4.7 supports a wide range of platforms. It can be deployed on iOS, Android, and HarmonyOS, with adaptations completed for Rockchip, Houmo, Intel, AMD, and Arm chip platforms on the day of release. Inference framework support includes SGLang, vLLM, llama.cpp, and Ollama. Fine-tuning tools include SWIFT and LLaMA-Factory. Quantized versions are available in GGUF, AWQ, GPTQ, FP8, and several other formats.

Behind this roadmap lies the Density Law proposed by ModelBest and Tsinghua University. According to this law, the capability density of large models doubles approximately every 3.5 months, meaning fewer parameters are required to achieve the same level of capability. As of September 2026, cumulative downloads of the MiniCPM series have exceeded 50 million.

In summary, carrying high-performance large models smoothly in your pocket is no longer a difficult feat.

Like, Share, and Favorite

We welcome your thoughts in the comments!

— End —

🌟 Star Us 🌟

Daily Updates on Cutting-Edge Tech Progress