Nvidia fits a 74-billion-parameter agent model into 24 GB of VRAM
On October 15, 2026, Nvidia launched Nemotron 3.5 Bolt, a 74-billion-parameter agent model compressed with 2-bit quantization to fit in 24 GB of video memory, runnable locally through llama.cpp, Ollama or LM Studio. If you want a local agent for coding or browsing, test it; extreme quantization remains a tradeoff you must measure on your own tasks.
October 15, 2026. Nvidia expands its Nemotron family with Nemotron 3.5 Super VL, a 120-billion-parameter vision-language model, and two Bolt models aimed at local execution. 74 billion parameters. That is the size of the larger Bolt, brought down to 24 GB of memory through 2-bit quantization. 9 billion active. Only 9 of the 74 billion parameters work at each inference, thanks to a mixture-of-experts architecture. Why it matters: for the first time, an agent model of this size fits in the memory of a consumer GPU, shifting the question from “can I run it locally?” to “what is it actually worth once compressed?”
An agent model built for the workstation
The Bolt models are explicitly designed for “everyday agentic tasks on laptops, balancing speed and intelligence needed to handle multistep reasoning, tool calling, and web browsing.” Two sizes are announced: Nemotron 3.5 Bolt 74B-A9B 24GB, available now, and Nemotron 3.5 Bolt E4B 4GB, due later this month and aimed at conversational and physical agents on GeForce RTX, Jetson, and Spark hardware.
The 74B model derives from Super VL and inherits its hybrid architecture — a combination of Mamba-2, attention, and mixture-of-experts layers — but without the vision encoder. It activates 9 billion of its 74 billion parameters at a time. To cut the memory footprint to 24 GB, Nvidia uses 2-bit quantization, in mixed precision with two-bit experts. That number is the whole story: 2 bits per weight is extreme compression, at the edge of what current formats allow before quality collapses. The model is optimized for RTX Spark and DGX Spark systems, with announced support for llama.cpp, Ollama, LM Studio, and vLLM — the four tools most people use to run a local model today.
Benchmarks that hold up against comparable models
In Nvidia’s evaluations, Bolt 74B-A9B scores 51.0 on Terminal-Bench 2.1, a command-line agent benchmark, against 44.4 for Qwen3.6-35B-A3B and 37.2 for Gemma4 26B-A4B, both of the latter tested in BF16. The model beats lighter, less-compressed rivals — which is the comparison that actually matters: the question is not whether it beats a giant hosted model, but what you get for 24 GB of VRAM on a desk.
Those vendor numbers deserve the usual caution — a benchmark measured by the seller, on its own tasks. But the order of magnitude is consistent with Nvidia’s thesis: extreme quantization, applied to a mixture-of-experts architecture that only activates a fraction of its weights, costs less quality than feared for agent work, because rarely-used experts tolerate reduced precision well.
The counterpoint comes from the larger model in the same release. Nemotron 3.5 Super VL — 120 billion parameters, 12 billion active, up to 1 million tokens of context, image and video input — posts mixed results against Qwen3.5-122B-A10B: better on scientific reasoning and terminal tasks (66.7 versus 48.7 on Terminal-Bench), but slightly behind on SWE-bench Verified (71.1 versus 71.9) and document understanding. The newer Qwen3.8-Flash-Next beats it on all five tests, at the cost of a higher total parameter count, around 180 billion. The general lesson: among open models of this size, nobody dominates everything, and the choice comes down to the specific task.
What “24 GB” actually changes
The real stake of the Bolt model is not a benchmark record but a shift in the hardware frontier. Until now, a reasonably capable agent model required either an expensive professional card or a call to a remote API — with the cost, latency, and privacy questions that implies. At 24 GB, the model fits in the memory of a high-end consumer card, and the bar drops to what many developers already own.
That has two direct consequences. The first is data sovereignty: an agent that browses, calls tools, and manipulates files can work entirely offline, with no code or documents leaving the machine. For proprietary code or sensitive data, that is a decisive argument against a hosted agent. The second is cost predictability: an already-amortized card runs inferences whose marginal cost is electricity, where a cloud agent bills per use and can see its tab drift upward over long multistep sessions.
The tradeoff is familiar to anyone who has run a quantized model: precision suffers on tasks that demand finesse — heavy mathematical reasoning, long code generation, strict format compliance. A 2-bit model can hallucinate more or fail on an edge case that a BF16 model of the same architecture would have passed. That is precisely what you must measure on your own tasks, not on a generic benchmark.
How to run it, and what to verify
The model is published on Hugging Face under the OpenMDW 1.1 license, which permits commercial use and modification — decisive for a team that wants to build it into a product rather than just try it out. On the runtime side, Ollama and llama.cpp already expose it, with the flow identical to any other local model except that the 2-bit quantization is baked into the published weights rather than being a step you apply yourself.
# Local execution via Ollama (once the model is on the registry)
ollama pull nvidia/nemotron-3.5-bolt:74b
ollama run nvidia/nemotron-3.5-bolt:74bThe real test is not the first prompt: it is a long, multistep agent session with tool calls and web browsing. That is where extreme quantization either pays off or fails — a model that reasons well on a benchmark but loses the thread at the twelfth turn of an agent session is little use in real conditions. Measure drift on a task you actually perform, not on a generic score.
This release also signals a broader shift. Local agent models are no longer a hobbyist niche: they are becoming a credible fallback for teams that cannot send code or customer data to a hosted API for compliance reasons, and a hedge against the price and latency of hosted agent sessions. Nvidia’s move is less about beating the largest hosted models and more about making the “run it yourself” path good enough that the default stops being “call the API.” The E4B 4GB variant, due later this month, pushes the same idea further down the hardware ladder, to devices where even 24 GB is a luxury.
Verdict
If you want a local agent for coding, researching, and browsing on sensitive data, Nemotron 3.5 Bolt 74B is a serious candidate: it fits in 24 GB, runs under the tools you already have, and its benchmarks beat comparable models — test it on your own agent tasks rather than on benchmarks. If your priority is maximum accuracy on demanding reasoning or code generation, stay on a less-compressed model or an API call to a large model, and accept the cost. If you do not have a 24 GB card, wait for the Bolt E4B 4GB due this month, or stick with quantized 7–14B models, which are often enough for conversation and simple tasks. The new frontier is no longer “is a local agent model possible?” — it is “how much quality are you willing to trade to stop sending your data elsewhere?”