Nemotron 3.5 Lightning matches a 120B model’s performance with 3.6B active parameters
On August 11, 2026, NVIDIA released Nemotron 3.5 Lightning, an open 31.6B-parameter MoE model (3.6B active) that matches a 120B model’s Intelligence Index while serving nearly 670 tokens per second. For high-volume agentic workloads, it is a concrete cost lever.
August 11, 2026. NVIDIA releases Nemotron 3.5 Lightning, the first model of the Nemotron 3.5 family. 31.6 billion total parameters, 3.6 billion active, an Intelligence Index of 24 — the same as gpt-oss-120b, a model four times its size — and nearly 670 tokens per second on a pre-release DeepInfra endpoint.
NVIDIA’s thesis is straightforward: you no longer need a frontier model for every call an agent makes. A small specialized model, routed intelligently, covers most high-volume tasks. It is, in effect, the argument that the agent economy splits into two tiers — expensive planning and cheap execution — and that the second tier is now a solved problem for open weights.
A +9 point jump without touching the architecture
Nemotron 3.5 Lightning succeeds Nemotron 3 Nano 30B A3B and keeps its skeleton: the hybrid Mamba-Transformer mixture-of-experts (MoE) architecture, with 3.6 billion active parameters out of 31.6 billion total. Nothing new on the structure side — all the gains come from post-training.
The result is still sharp. On Artificial Analysis’s Intelligence Index, the model moves from 15 (Nano) to 24, a +9 point jump, landing level with gpt-oss-120b (24) and just behind Nemotron 3 Super (26), a model roughly four times larger. NVFP4 quantization is near-lossless: 24 on the index versus 24 in BF16.
That is the most interesting signal for open AI right now: progress no longer comes from raw size, but from the quality of post-training on a compact, efficient base.
The agentic jump
The most dramatic gain shows up on agentic tasks. On GDPval-AA v2, the model reaches an Elo of 824, surpassing both Nemotron 3 Super and gpt-oss-120b. On Terminal-Bench v2.1, it climbs to 24%, more than three times Nemotron 3 Nano’s 7%, and nearly level with gpt-oss-120b.
Concretely: a model the size of a “small” one executes code review, tool calls, security alert triage, and billing answers with the reliability once reserved for frontier models. NVIDIA post-trained the release with partners like CodeRabbit and Harvey to strengthen domain performance — ease of training being exactly the family’s selling point.
The context window reaches 1 million tokens, opening long workloads: full-repository review, multi-step agent sessions, large log follow-up.
Speed changes the economics
The metric that separates a demo model from a production model is time per task. Nemotron 3.5 Lightning handles an Intelligence Index task in about 0.5 minutes — versus ~3.5 minutes for Qwen3.6 35B A3B, ~3.4 minutes for gpt-oss-120b, and ~5.8 minutes for Gemma 4 31B.
That speed has two engines: high output throughput — nearly 670 tokens/s measured on the pre-release DeepInfra endpoint — and token efficiency comparable to Nano. The model emits as many tokens per task as its predecessor while delivering +9 points of intelligence.
For an operator billed by usage, the equation is direct: a model that responds seven times faster than a comparable-size rival, at equal or better quality, divides the workload’s bill by a factor of the same order. The OpenMDW-1.1 license, permissive with no material commercial restriction, removes the last barrier to self-hosting.
Where Lightning sits in the Nemotron family
Nemotron 3 ships in three sizes: Nano (small), Super (medium), and Ultra (large). Lightning slots in as a specialized variant of the small end: it keeps Nano’s footprint but targets a specific use — high-volume specialized task execution inside long-running agents.
The MoE logic is at the heart of the argument. Of 31.6B parameters, only 3.6B are active per token. The model therefore stores the knowledge of a large model but computes only a fraction of it at each inference step. That is what enables both low inference cost and high throughput, without sacrificing knowledge capacity.
NVIDIA pushes this logic all the way by publishing FP8 and NVFP4 quantizations that are near-lossless, and by shipping the training recipes — not just the weights. The stated goal: let each team post-train the model on its own tasks, turning it into a bespoke executor.
For a self-hoster, the math sharpens further. An NVFP4 model of this size fits on a single modest GPU, where a 120B demands several accelerators. The saving is not only per-token cost but the hardware required to serve the load locally — and therefore the threshold at which self-hosting beats serverless.
Switchyard: routing to the right model
NVIDIA does not position Lightning as a standalone model. It plugs into NeMo Switchyard, an intelligent routing mechanism between models: high-volume tasks — specialized execution, code review, alert monitoring — go to Lightning, while complex planning stays with frontier models.
This is the architectural pattern spreading through agentic systems: an expensive, powerful planner, a compact, fast executor, and a router in between. Lightning occupies exactly the executor slot — where call volume dominates and marginal cost matters most.
Weights are available now, in BF16, FP8, and NVFP4, with serverless inference on DeepInfra, Fireworks, FriendliAI, CoreWeave, GMI Cloud, Nebius, and Crusoe.
Assumed limits
The model remains text-only — no image or audio — and on raw intelligence, comparable-size competitors beat it: Qwen3.6 35B A3B (32) and Muse Glimmer (35) on the Intelligence Index. Proprietary models also keep the edge on the overall efficiency frontier: Gemini 3.5 Flash-Lite reaches 37 at a comparable time per task.
Lightning is not trying to win the raw-intelligence leaderboard. It optimizes a precise point on the frontier: intelligence per dollar and per second on specialized agentic tasks. That is a positioning choice, not a weakness — but it must be read as such before adopting it for generalist reasoning workloads.
Being text-only is a real constraint for some pipelines. If your agents reason over screenshots, documents, or audio, Lightning alone will not replace a multimodal stack — it sits upstream of perception, taking structured text and turning it into decisions and tool calls. Pairing it with a separate vision or speech model is the expected design, not a workaround.
The competitive picture is clear: Lightning is not the smartest open model of its size, but it is among the fastest and cheapest to run at its quality tier. For teams already routing agent traffic through Switchyard or a similar gateway, the question is not whether to replace a frontier model but whether to stop paying frontier prices for executor-tier calls.
Verdict
If you run high-volume agentic workloads — code review, tool calls, alert triage, repetitive domain questions — Nemotron 3.5 Lightning deserves an immediate trial through a serverless provider, before even considering self-hosting. Its quality-to-speed-to-cost ratio on those tasks is among the best in the open market today.
If you need generalist or multimodal reasoning, stay on larger or multimodal models: Lightning is a specialized executor, not a generalist planner. Paired with Switchyard it comes into its own; alone, it will leave you wanting outside its home turf.
The underlying signal: the race is no longer about parameter count, but about post-training and routing. A model with 3.6B active parameters matching a 120B on agentic tasks, at a seventh of the time, redraws the map of what is worth self-hosting.