NVIDIA opens four Cosmos 3 models for interactive world simulation and multi-sensor driving scenes
On November 5, 2026, NVIDIA added four open-weight checkpoints to its Cosmos 3 family, able to stream synchronized camera-LiDAR driving scenes and predict video from robot actions. Robotics and autonomous-driving teams now have an open starting point to simulate — and then adapt to their own sensor rig.
November 5, 2026. NVIDIA adds four open-weight checkpoints to its Cosmos 3 family, aimed at interactive world simulation and multi-view, multi-sensor driving scenes. 16 billion. The number of trainable parameters in each of the four models, all post-trained from Cosmos 3 Nano. OpenMDW 1.1. The license permits commercial as well as non-commercial use. Why it matters: the world model — the environment that answers an agent’s actions — stops being an industrial secret and becomes an open starting point, re-adaptable to your own robot or vehicle.
Four checkpoints, four ways to close the loop
The whole announcement turns on one distinction that sums up the modern simulation problem. A video generator produces images; a world model produces a consequence. The value is not in the rendering but in the fact that each observation can shape the next action.
The four models cover both ends of that loop.
Cosmos3-Nano-Transfer-Auto is a bidirectional model for offline driving-data generation. It produces video synchronized across up to eleven camera views — including fisheyes — and supports joint RGB-LiDAR generation. Per-camera World Scenario Map (WSM) controls specify scene structure and traffic motion, while text prompts guide appearance. It is also the starting point for adapting Cosmos to your own sensor: you post-train it on your data to accommodate your camera layout, LiDAR configuration, and target environment.
Cosmos3-Nano-Sim-Auto is the causally distilled streaming variant of Transfer-Auto. It generates small successive chunks of synchronized camera and LiDAR video in just a few denoising steps, relying on already-generated history and updated driving controls. This is the model that enables long, interactive rollouts — the one that turns a generator into a simulator.
Cosmos3-Nano-Sim-Transfer is for those who already have a physics simulator. The simulator computes motion, contacts, and physical state; Sim-Transfer takes its spatial controls and a text prompt to render photorealistic color observations. You do not replace the physics engine — you give it a generative renderer.
Cosmos3-Nano-Sim-Bimanual closes the list on the robotics side: it predicts video from robot or camera actions, opening the door to bimanual manipulation without a physical environment.
The post is not just weights. NVIDIA ships the practical plumbing alongside: an open skill and a notebook for running Transfer-Auto and Sim-Auto, so a researcher can go from download to a first generated scene in a single session rather than reimplementing the inference stack. That matters because the bottleneck in world-model adoption has rarely been model quality — it has been the integration cost of turning a checkpoint into a runnable, controllable generator. Lowering that bar is as much a part of the release as the 16 billion parameters.
The real leap is not rendering, it is streaming
If one checkpoint deserves the attention, it is Sim-Auto. Causal, streaming generation changes the nature of the tool. A classical generator emits a sequence in one block, with no feedback. Sim-Auto emits successive chunks conditioned on history and incoming controls, letting a driving policy run in closed loop: the agent acts, the world responds, the agent acts again.
That is exactly the property video-generation models lacked for robotics and autonomous driving. NVIDIA says it plainly in the post: streaming generation “connects video prediction to ongoing interaction,” giving world-model researchers a way to study how actions and observations unfold together. For a vehicle, appearance and geometry must also agree: synchronized camera-LiDAR generation links visual observations to 3D sensing, while scene and traffic-signal controls let researchers vary road layouts and signal states to study how a policy responds.
The four-stage training recipe lays out the full industrial path: post-train Transfer-Auto on your data, then distill it into a few-step model to obtain Sim-Auto. The message is clear — NVIDIA is not just shipping weights, it is shipping an adaptation playbook, accepting that every lab has a different sensor and domain.
An open foundation, with limits owned
The announcement is remarkably honest about what it does not solve. NVIDIA concedes it up front: “these problems remain hard.” The work ahead is to measure where the models faithfully support policy learning and evaluation, understand their limits, and improve where they fall short. Open weights and post-training recipes offer a shared starting point for adapting, comparing, and contributing — not a turnkey solution.
That frankness stands out from the sector’s usual messaging. Opening the weights under OpenMDW 1.1, with commercial use allowed, is also a strategic calculation: the more labs build on Cosmos, the stronger the NVIDIA hardware platform — the GPUs that train and serve these models — becomes. It is not philanthropy, but it is an alignment of interests the open-source ecosystem benefits from. The NVIDIA line in this space is already well documented on this blog, from the open-weight Nemotron post-training to the Kumo tabular models.
What “interactive simulation” changes for embodied AI
To see why these checkpoints matter, place the world model in the trajectory of embodied AI. A robot or a vehicle learns a policy by trying actions and observing their consequences. The real world is too slow, too expensive, and too dangerous to supply those trials in volume, so simulation is the default training ground. But a fixed simulation only covers the scenarios you scripted in advance. A generative world model inverts the constraint: instead of writing every scenario, you sample worlds.
The four models fit that logic. Sim-Bimanual predicts the visual consequences of actions for manipulation robotics; Sim-Transfer renders an existing physics simulator photorealistically, letting you generalize appearance without rewriting the physics. The World Scenario Map — those per-camera controls describing the scene and traffic — acts as a script: it says what must exist in the scene, and the model handles the rest.
Opening the weights also changes reproducibility. A closed world model is judged on demos; an open one is judged on replayable benchmarks, independent post-training, and method comparisons. It is the same movement open-weight LLMs went through: opening the weights shifts progress from the vendor to the ecosystem. NVIDIA accepts this explicitly by publishing the four-stage recipe — a transfer of know-how that benefits the hardware platform, but above all makes the category measurable.
Verdict
If you are building a simulation stack for autonomous driving, start from Transfer-Auto before writing a line of your own generator. The shortest path to a simulation adapted to your vehicle is to post-train on your own sensor data, then distill into Sim-Auto for interactive streaming — precisely the recipe NVIDIA documents. On the robotics side, Sim-Bimanual and Sim-Transfer cover your two cases: predicting the consequence of an action, or rendering an existing physics simulator photorealistically. If you are only observing, take the signal: the interactive, open-weight world model is now a public artifact, and the distinction that matters is no longer “generative or not” but “closed-loop or not.” The value has moved from rendering to responsiveness to actions — and that is where the next generation of simulators will be decided.