Reka Launches Rho-1, a 19B Omni-Model Handling Text, Images, Video and Robot Actions in One Network

What Happened

Reka AI — founded by ex-Google DeepMind senior scientist Dani Yogatama, which shipped the GPT-4-class Reka Core in April 2024 — launched Rho-1 via an official blog post on Monday, with THE DECODER and RuntimeWire covering the same day. The demo is a single five-turn session: draw a red-and-white lighthouse coast scene, box the lighthouse, animate it into a drone-approach video, restyle the weather into a snowstorm, then answer 'what changed between the two videos' — all inside the same model, the same KV-cache state, with zero tool calls and no second model.

Architecturally, Rho-1 is 'symmetric': inside every transformer block, two expert weight streams — one for understanding (language and visual parsing), one for generation (image/video denoising) — sharing attention and one KV cache; discrete tokens encode text, reasoning and commands, while continuous tokens encode image latents, video frames, robot actions and proprioception. Because inputs and outputs share the same formats, anything the model generates feeds straight back into context — predicting the next frame, the next instruction and the next action is, in Reka's words, 'one operation, not three models sharing a bus.'

Key Facts

1. Speed numbers and the chart legend: The base model generates video at a median 0.79x real time, with a watchable stream starting in roughly 6 seconds; the distilled variant cuts the denoising trajectory from 99 steps to 8, rendering a 5.3-second clip in about a second — 'faster than any video model we timed,' the company says. The blog also shows a comparison: a 13.8-second multi-agent pipeline route versus Rho-1's measured 7.0 seconds. Per the legend, the pipeline bar is labeled 'illustrative' — only Rho-1's 7.0 seconds is measured.
2. Training: 320 H100s, about 3 months: Rho-1 is a symmetric-architecture model trained from scratch. The blog acknowledges that multi-objective training (generation and understanding without dragging each other down) has no settled consensus and that training instability is an open research problem.
3. Robot action data: the Inverse Dynamics Model: Robotics lacks action data — teleop logs are expensive and embodiment-specific. Reka addresses this with its Inverse Dynamics Model: ordinary internet video doesn't record steering angles or joint torques, so the IDM observes raw video, infers the underlying control signals, and feeds those actions to Rho-1 as native tokens. The demos show a banana lifted off a plate, a mug carried by its handle, and an apple picked from the wrist camera's view, under pure text conditioning.
4. Reka's own Limitations section: The blog carries a standalone Limitations chapter: long-horizon drift (a 30-second stream keeps photorealistic texture while drifting into a structurally incompatible room layout), unreliable object grounding across video, brittle targeted editing across prompts, and native video capped at 672x384.
5. Availability: research preview: This is a research preview — no released weights, no API, no third-party benchmarks. Every speed number is company-tested, and the demo scenes were chosen by the company.

Context

Google DeepMind has Genie 3 (text-generated, explorable real-time environments); Reka's Rho-1 puts simulation, reasoning and action into a single network. In September, Reward AI released OM-1, which trains robot policies on human demonstration data. Reka's previous flagship was the language model Reka Core (April 2024, benchmarked against GPT-4 / Claude 3 / Gemini Ultra); Rho-1 is its move into physical intelligence, using a symmetric architecture and an inverse-dynamics data pipeline.