NVIDIA Open-Sources Nemotron 3.5 Super VL: 120B MoE Model Boosts Agent Throughput 5x on Single GPU

On October 15, NVIDIA announced the release of Nemotron 3.5 Super VL, an open multimodal model designed for AI agents that require continuous operation and multi-step task execution.

The model utilizes a Mixture-of-Experts (MoE) architecture with a total parameter count of 120 billion and 12 billion active parameters. NVIDIA claims that when executing agent tasks on a single GPU, this model can achieve up to 5 times the throughput of other open models of comparable scale.

According to the introduction, Nemotron 3.5 Super VL integrates reasoning, instruction following, and tool-calling capabilities with text, image, document, and video understanding into a single model. It is suitable for scenarios such as code testing and debugging, cross-site information retrieval, document analysis, cybersecurity threat identification, and video content search. Developers can leverage the NVIDIA NeMo toolkit to perform post-training using their own data, tools, policies, and workflows, adjusting model behavior according to business needs.

NVIDIA also disclosed several enterprise customization cases. For example, AI data technology provider Scale AI customized a vendor credit processing agent for media entertainment company Skydance Corporation. The inference cost for related workloads is expected to decrease by 97.5%, with latency reduced by 80%.

Financial operations platform Ramp is converting receipt images into structured JSON data containing merchant, date, and payment information, expecting up to a 50% increase in throughput and a 90% reduction in latency. AI21 collaborated with AI security company Alice to customize models for threat intelligence and vulnerability analysis, achieving a 40% improvement in accuracy for related tasks.

In terms of model evaluation, NVIDIA stated that Nemotron 3.5 Super VL scored 80.8% on the CharXiv chart reasoning benchmark, higher than Nemotron 3 Nano Omni's 63%; when combined with Python code interpretation, the score reached 84%.

On Hermes Bench and LangChain behavioral evaluations, the model achieved scores of 63.44% and 87.05%, respectively, surpassing Nemotron 3 Ultra's 56.33% and 84.3%. NVIDIA also noted that the model ranks among the top-performing open models on the OCRBench v2 leaderboard.

To reduce inference overhead, Nemotron 3.5 Super VL employs techniques such as speculative decoding, mixed precision, and context reuse. NVIDIA introduced that its LiLiCorr speculative decoding method can boost token generation speed by up to 2.3 times on DGX Spark. The model also supports NVFP4 quantization, using 4-bit weights and 4-bit activations on NVIDIA Blackwell architecture; on non-Blackwell platforms, the same checkpoint uses 4-bit weights and 16-bit activations.

NVIDIA also unveiled deployment optimization strategies for distributed inference. According to disclosed data, under test conditions using 16 NVIDIA B200 GPUs with a generation rate of 50 tokens per user per second, NVIDIA Dynamo—combined with cache-aware routing, session affinity, and KV cache offloading—can serve up to 8 times more users while reducing time-to-first-token latency by 85%.

This release includes base models, post-trained models, and NVFP4 quantized model weights, along with training recipes, datasets, and reinforcement learning environments.

NVIDIA stated that Nemotron 3.5 Super VL utilized approximately 26 trillion tokens during pre-training and continued pre-training phases, about 3.8 million samples in the post-training phase, and underwent reinforcement learning across dozens of environments. The newly released datasets cover areas such as agentic tool calling, financial reasoning, instruction following, long-context handling, fairness, and visual-language understanding.

In addition to Super VL, NVIDIA introduced two Nemotron 3.5 Bolt edge-side models. The Nemotron 3.5 Bolt 74B-A9B 24GB employs a Mixture-of-Experts (MoE) architecture with 74 billion total parameters and 9 billion active parameters. It is compressed to 24GB via native 2-bit quantization, targeting local agent tasks, and supports llama.cpp, vLLM, Ollama, Unsloth, and LM Studio. The Nemotron 3.5 Bolt E4B 4GB is designed for scenarios such as local voice assistants, interactive characters, and embedded interfaces, with a planned launch on October 23.

IT Home noted that Nemotron 3.5 Super VL and Bolt 74B-A9B 24GB are now available. Developers can access these model resources through channels such as Hugging Face, OpenRouter, and ModelScope. Super VL can be deployed on DGX Spark, DGX Station, RTX systems, data centers, or the cloud; Bolt E4B 4GB is scheduled for release on October 23.