NVIDIA Releases Four Cosmos 3 Open Weights: 16B World Models for Driving Data and Robot Simulation

How it unfolded

The release came in two steps. On Nov 2, NVIDIA's technical blog described an expansion of the Cosmos platform: Cosmos Framework now extends beyond Cosmos 3 to cover Alpamayo, GR00T, Qwen, pi0.5 and other compatible models, adding reinforcement-learning pipelines including DPO, GRPO and PPO plus Cosmos NIM deployment containers; on Nov 5, the four checkpoints went live on Hugging Face, and NVIDIA hosted a Cosmos Labs livestream the same day to demonstrate them. Cosmos is NVIDIA's platform of world foundation models and development tools for physical AI, with the Framework supplying training, optimization and inference recipes and NIM supplying GPU-accelerated deployment containers.

Two checkpoints target autonomous driving. Transfer-Auto is a bidirectional model for offline synthetic driving data: it generates synchronized video across up to 11 camera views, including fisheyes, plus LiDAR point clouds, with developers setting road layout and traffic behavior through per-camera World Scenario Map (WSM) controls and steering appearance with text prompts; it supports 30 fps input and output video, with up to 400 frames per segment at 480p or 720p and longer sequences built by autoregressive chunking. Sim-Auto is the causally distilled streaming variant of Transfer-Auto, generating each small chunk of synchronized camera video and LiDAR in a few denoising steps and, paired with a simulation runtime and WSM renderer, feeding a driving policy's actions back into the scene for long interactive rollouts and closed-loop evaluation.

The other two target robotics. Sim-Transfer turns a physics simulator's controls and a text prompt into photorealistic RGB video: four spatial controls — depth, segmentation, edges and blur — keep the physical state in the simulator while text guides appearance, and a policy consumes the generated frames directly. Sim-Bimanual predicts the next video frame from bimanual robot or camera actions, supporting interactive two-arm simulation for the YAM and AgiBot embodiments; one control sequence can switch between viewpoint changes and manipulation, and the mixed-action example uses 59 control values (padded to 64 during training). The blog also covers surgical simulation: Cosmos3-H-Embodied-Nano can generate surgical video from robot actions, predict actions from video, or do both together, while Cosmos3-H-Dreams is a distilled variant for real-time closed-loop simulation.

The training and runtime figures were published alongside. All four checkpoints share a four-stage training recipe: bidirectional adaptation, teacher forcing, consistency distillation (four denoising steps per chunk) and self-forcing, with the final stage training on the model's own generated history to slow error accumulation; the post-training recipes ship with the inference weights so developers can adapt the models to their own sensor rigs and embodiments. The backbone is a Mixture-of-Transformers, connecting text-processing and visual-generation networks through attention. Running the models requires Linux, an NVIDIA GPU, Docker and the NVIDIA Container Toolkit; NVIDIA's technical blog lists 58 GiB in BF16, 44 GiB in FP8 and 31 GiB with FP8 plus offloading for Cosmos 3 Nano generation. Demonstration figures in the technical report put Sim-Bimanual at 48.1 fps with 166 ms latency at 480p on a single GB300.

What was said

The Hugging Face blog frames the release as a shared starting point for researchers and developers, saying open weights and post-training recipes allow adapting models, comparing methods and contributing improvements; it also states that the work ahead is to measure where the models faithfully support policy learning and evaluation, understand their limits, and improve where they fall short.

These problems remain hard: the work ahead is to measure where the models faithfully support policy learning and evaluation, understand their limits, and improve where they fall short.

— Hugging Face official blog (NVIDIA Cosmos team)

NVIDIA's technical blog says physical AI developers face two challenges: adapting models to their sensors, tasks and data, and getting those models into applications reliably; the Cosmos platform addresses both, with the Framework covering the full development lifecycle of training, reinforcement learning, optimization and inference, and NIM handling accelerated deployment. The blog also says Cosmos Framework is expanding beyond Cosmos 3 to more compatible models, with inference supported on PyTorch and vLLM-Omni and tested on Hopper and Blackwell GPUs.

Independent coverage flags the release's limits: the blog is partnership content hosted on Hugging Face, no one outside NVIDIA has independently measured the visual quality of these checkpoints, and NVIDIA has not released evaluation code or error bars, so the demonstration figures describe specific hardware configurations. The same coverage notes that releasing four checkpoints at once gives physical AI teams different entry points rather than a single demonstration model.

Key facts

1. Publisher and date: NVIDIA, Nov 5, 2026; the checkpoints were published on Hugging Face, with the technical blog posted Nov 2 and a Cosmos Labs livestream the same day as the release
2. The four checkpoints: Cosmos3-Nano-Transfer-Auto, Cosmos3-Nano-Sim-Auto, Cosmos3-Nano-Sim-Transfer and Cosmos3-Nano-Sim-Bimanual, all post-trained versions of Cosmos 3 Nano
3. Parameters: Each has 16 billion trainable parameters; the backbone is a Mixture-of-Transformers, connecting text-processing and visual-generation networks through attention
4. License: OpenMDW1.1, permitting commercial and non-commercial use under its terms
5. Transfer-Auto: Offline synthetic driving data, generating synchronized video across up to 11 camera views including fisheyes, plus LiDAR point clouds; per-camera World Scenario Map controls set road layout and traffic behavior; 30 fps, up to 400 frames per segment (480p or 720p)
6. Sim-Auto: The causally distilled streaming variant of Transfer-Auto, generating each small chunk of synchronized camera video and LiDAR in a few denoising steps for long interactive rollouts and closed-loop evaluation
7. Sim-Transfer: Simulator controls plus a text prompt become photorealistic RGB video; depth, segmentation, edge and blur spatial controls keep the physical state in the simulator
8. Sim-Bimanual: Predicts the next video frame from bimanual robot or camera actions; supports YAM and AgiBot embodiments, controlled by a policy or human-operated joysticks; the mixed-action example uses 59 control values; demonstration figures in the technical report are 48.1 fps and 166 ms latency at 480p on a single GB300
9. Training recipe: Four stages — bidirectional adaptation, teacher forcing, consistency distillation and self-forcing; post-training recipes and inference weights are released together for adaptation to new sensor rigs and embodiments
10. Requirements and limits: Running the models requires Linux, an NVIDIA GPU, Docker and the NVIDIA Container Toolkit; Cosmos 3 Nano generation needs 58 GiB in BF16, 44 GiB in FP8 and 31 GiB with FP8 plus offloading; NVIDIA says it remains to be measured whether the models reliably support policy learning and evaluation