NVIDIA Releases Four Cosmos 3 World Models: 16B Open Weights for Driving and Robot Simulation

What happened

The four checkpoints split by simulation task. Transfer-Auto targets offline driving data generation, producing synchronized video across up to 11 camera views, including fisheyes, plus LiDAR point clouds; developers set road layout and traffic behavior with per-camera World Scenario Map controls and steer appearance with text prompts. Sim-Auto is the causally distilled streaming variant of Transfer-Auto, generating successive small chunks of synchronized camera video and LiDAR for long interactive rollouts.

The other two target robotics. Sim-Transfer turns a physics simulator's controls and a text prompt into photorealistic RGB video, using four spatial controls — depth, segmentation, edges and blur — while the simulator keeps the physical state and text guides appearance. Sim-Bimanual predicts the next video frame from bimanual robot or camera actions, supporting interactive two-arm simulation for the YAM and AgiBot embodiments; one control sequence can switch between viewpoint changes and manipulation, and the mixed-action example uses 59 control values.

NVIDIA also published the four-stage training recipe — bidirectional adaptation, teacher forcing, consistency distillation and self-forcing, with the final stage training on the model's own generated history to slow error accumulation — alongside the post-training recipes so developers can adapt the models to their own sensor rigs and embodiments. NVIDIA's technical blog of Nov 2 frames the platform as serving autonomous driving, robotics and smart infrastructure, and the company hosted a Cosmos Labs livestream on Nov 5; NVIDIA also says researchers still need to measure whether the models reliably support policy learning and evaluation.

Key facts

1. Publisher and date: NVIDIA, Nov 5, 2026; the checkpoints were published on Hugging Face, with the technical blog posted Nov 2
2. The four checkpoints: Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer and Cosmos3-Nano-Sim-Bimanual, all post-trained versions of Cosmos 3 Nano
3. Parameters: each has 16 billion trainable parameters; the backbone is a Mixture-of-Transformers, connecting text-processing and visual-generation networks through attention
4. License: OpenMDW1.1, permitting commercial and non-commercial use under its terms
5. Transfer-Auto: offline synthetic driving data, generating synchronized video across up to 11 camera views including fisheyes, plus LiDAR point clouds; per-camera World Scenario Map controls set road layout and traffic behavior
6. Sim-Auto: the causally distilled streaming variant of Transfer-Auto, generating each small chunk of synchronized camera video and LiDAR in a few denoising steps for long interactive rollouts
7. Sim-Transfer: simulator controls plus a text prompt become photorealistic RGB video; depth, segmentation, edge and blur spatial controls keep the physical state in the simulator
8. Sim-Bimanual: predicts the next video frame from bimanual robot or camera actions; supports YAM and AgiBot embodiments, controlled by a policy or human-operated joysticks; the mixed-action example uses 59 control values
9. Training recipe: four stages — bidirectional adaptation, teacher forcing, consistency distillation and self-forcing; post-training recipes and inference weights are released together for adaptation to new sensor rigs and embodiments
10. Requirements and limits: running the models requires Linux, an NVIDIA GPU, Docker and the NVIDIA Container Toolkit; NVIDIA says it remains to be measured whether the models reliably support policy learning and evaluation