Joint WAM
Chunk Time 667.1 msTotal Time 68 s
Abstract
World Action Models (WAMs) jointly generate future visual observations and robot actions, allowing policies to reason about how the scene may evolve under interaction. Their iterative generation, however, is often slower than the robot control cycle: synchronous execution leaves the robot idle during inference, while naive asynchronous switching can create inconsistency between successive predictions.
We introduce Stream-WAM, an action-conditioned streaming framework that overlaps WAM inference with robot execution. Shared actions across adjacent chunks condition future video generation, aligning the predicted visual trajectory with the motion underway; this action-conditioned future then guides a consistent action continuation. Stream-WAM therefore brings streaming into world prediction rather than treating continuity only as an action space constraint. We evaluate the method on LIBERO, RoboCasa, and RoboTwin 2.0. On LIBERO, Stream-WAM achieves 98.20% success with 41.0 ms chunk latency and 5.36/3.15 s total time on Long/Short tasks, yielding a 12.0× latency reduction and 3.0×/2.6× total-time speedups over FastWAM.
World Action Models and the Real-Time Gap
World Action Models (WAMs) jointly predict future visual representations and action chunks, allowing robot policies to connect action generation with object motion, physical interaction, and task progress [1]. The visual branch provides a temporally structured account of how the scene may evolve, while the action branch turns that prediction into a multi-step control plan. This coupling has produced strong manipulation performance, but iterative generation over both modalities is often much slower than the robot control cycle.
Under synchronous deployment, inference and execution are serialized: the robot finishes its current chunk, waits for the next prediction, and only then resumes motion. Model latency therefore becomes idle time or repeated stale commands. Because the observation that launched inference is also aging during this pause, recent changes in object state, contact, or robot motion cannot influence the policy until the next update.
Longer action horizons reduce the frequency of model calls but commit the robot to a staler open-loop plan. Faster inference shortens the wait, yet any remaining latency still lies on the critical path. The deployment gap is therefore not only a question of model speed, but also of how prediction and execution are scheduled. Real-time WAM deployment requires prediction to overlap ongoing motion without losing consistency between the predicted world evolution and the actions actually executed.
Asynchronous Execution Beyond Action Continuity
Asynchronous execution starts the next prediction before the current action chunk is exhausted. This creates a temporal overlap in which the same actions are shared across adjacent chunks, and the incoming prediction must remain consistent with the motion already underway. Accurate temporal alignment is therefore essential: even individually valid chunks can be joined at the wrong steps if the handoff time is misidentified [2].
Real-time chunking (RTC) constrains the incoming chunk at inference time through inpainting-style guidance of the denoising trajectory [3]. Because this guidance is recomputed at every denoising step, it adds inference cost; as a soft constraint, it can also leave residual disagreement in the delay region [2]. Prefix-conditioned methods move the constraint into learned generation: delay-region actions are provided as clean context, and Training-Time RTC learns to produce the remaining action continuation from that context [2,4]. When runtime latency falls outside the delay distribution represented during training, the clean context may no longer match the actual handoff [4].
Both approaches align action trajectories rather than predicted world evolution. RTC steers the overlap between adjacent chunks, while prefix conditioning fixes the known action segment, but neither explicitly tells the visual branch how the shared actions transform the scene. This distinction matters in a WAM because the next action is generated together with a predicted account of the world. LingBot-VA 2.0 addresses the visual state through FDM grounding. It refreshes the visual cache with the latest observation, appends the executing action, and predicts that action's visual outcome as context for decoding the next action [5]. This extra grounding pass adds future-state prediction before each action update, a practical cost at high control rates.
Stream-WAM instead couples action continuity with world prediction inside a joint WAM. It reuses shared actions across adjacent chunks and routes them both directly into each Stream Update and into shared action slots. Together with the unknown action slots, the resulting condition slots guide future visual generation in the same joint update that produces the next visual-action chunk, while robot execution continues.
Stream-WAM aligns world prediction with continuous robot execution.
Stream-WAM is a streaming formulation for joint World Action Models. It overlaps model inference with robot execution and uses the actions scheduled during that overlap to condition the next prediction. Instead of pausing at every chunk boundary, the robot continues moving while the model prepares the next visual and action chunk. Because the model knows what motion will occur during inference, its world prediction remains aligned with ongoing execution rather than relying on the observation alone.
At the beginning of an episode, the WAM predicts the first visual and action chunk from the current observation, and the robot starts executing it. Before that chunk is exhausted, a fresh observation starts the next Stream Update in the background. The robot keeps following the current commands while the next chunk is generated. The handoff can be summarized as Atshared = At+1prefix: actions executed during inference are reused as shared context at the beginning of the incoming chunk. With an eight-action overlap, for example, A0[8:16] = A1[0:8]. When the new chunk is ready, the controller continues from the actions that follow this shared part rather than repeating them after the handoff. Each update follows the same loop, so ongoing execution provides both the time and the motion context needed for the next prediction.
A method diagram showing model updates overlapped with continuous robot execution. Shared actions connect the current and incoming chunks, enter both the action and condition paths, and condition future visual prediction through directed attention.
Shared actions serve two complementary roles. In action generation, they anchor the beginning of the incoming chunk so that the predicted continuation joins the current motion smoothly. In world prediction, an added attention path makes the same action context available to future visual generation. At the model level, this joint prediction can be written as (V̂t+1, At+1postfix) = F(Ot, Atshared): the visual future and action postfix are produced together from the current observation and the shared actions. If the shared actions close the gripper, for example, Stream-WAM jointly predicts the resulting scene transition and the actions that should follow it. The overlap is therefore aligned not only in the action sequence, but also in the predicted world state used for the next decision.
To make this overlap effective, each Stream Update must complete before the controller reaches the handoff. We use one-step consistency distillation to compress iterative joint visual and action denoising into a single denoising step. This removes repeated denoising calls from the streaming loop and makes ongoing execution a practical inference window rather than only a temporal alignment mechanism.
Condition slots give the action context a stable structure, summarized as Ct+1 = [E(At+1[0:k]), Ut+1[k:N]], where C is the condition-slot sequence, k is the number of shared slots, N is the total number of condition slots, E encodes the shared action prefix, and U denotes the unknown slots. The slot encoder removes unavailable action content and adds a learned state embedding that distinguishes known slots from unknown ones. This prevents placeholders from being treated as commands and helps the visual branch use only valid action context. Together, the shared action prefix and the condition slots connect continuous execution to action conditioned world prediction inside the same joint update.
Task performance. To evaluate Stream-WAM across WAM families, we further train FastWAM-Joint with the streaming approach on LIBERO and apply the same design to StarWAM on RoboTwin 2.0 and X-WAM on RoboCasa. All evaluations use four NVIDIA H100 GPUs.
We compare against general purpose robot policies and WAM baselines on task performance, and against WAM baselines on inference efficiency. CD denotes one-step consistency distillation. On LIBERO, we also ablate action conditioning and the slot encoder to assess each component. Best and second best task results are shown in bold and underlined, respectively.
LIBERO
LIBERO evaluation covers four suites: Long, Spatial, Goal, and Object, with 10 tasks per suite and 50 trials per task. We report average success across suites; Episode Time is reported separately for Long and Short tasks in the efficiency results.
| Method | Long | Spatial | Goal | Object | Average ↑ |
|---|---|---|---|---|---|
| OpenVLA | 53.7 | 84.7 | 79.2 | 88.4 | 76.5 |
| π₀ | 85.2 | 96.8 | 95.8 | 98.8 | 94.1 |
| π₀.₅ | 92.4 | 98.8 | 98.0 | 98.2 | 96.9 |
| Motus | 97.6 | 96.8 | 96.6 | 99.8 | 97.7 |
| Fast-WAM | 95.2 | 98.2 | 97.0 | 100.0 | 97.6 |
| FastWAM-Joint-CD | 97.20 | 99.60 | 98.60 | 100.00 | 98.85 |
| FastWAM-RTC | 58.40 | 76.20 | 77.00 | 83.40 | 73.75 |
| Stream-WAM (Ours) | 96.60 | 98.80 | 97.40 | 100.00 | 98.20 |
| Stream-WAM w/o Action Conditioning | 94.40 | 96.40 | 96.60 | 97.60 | 96.25 |
| Stream-WAM w/o Slot Encoder | 95.60 | 98.40 | 96.80 | 99.80 | 97.65 |
RoboTwin 2.0
RoboTwin 2.0 evaluates 50 tasks with 100 rollout episodes per task. Clean reports the easy setting and Random reports the hard domain-randomization setting.
| Method | Clean ↑ | Random ↑ | Total ↑ |
|---|---|---|---|
| π₀ | 65.92 | 58.40 | 62.2 |
| π₀.₅ | 82.74 | 76.76 | 79.8 |
| Motus | 88.66 | 87.02 | 87.8 |
| Motus from WAN2.2 | 77.56 | 77.00 | 77.3 |
| FastWAM-Joint | 87.8 | 87.32 | 87.56 |
| StarWAM-Joint | 84.8 | 86.0 | 85.4 |
| StarWAM-CD | 79.0 | 79.2 | 79.1 |
| Stream-WAM (Ours) | 87.2 | 88.8 | 87.6 |
RoboCasa
RoboCasa follows the standard 24-task protocol, with 50 trials per kitchen manipulation task and average success reported across tasks.
| Method | Average Success ↑ |
|---|---|
| π₀.₅ | 41.4% |
| π₀-FAST | 61.2% |
| π₀ | 62.5% |
| Cosmos Policy | 67.1% |
| X-WAM | 75.42% |
| X-WAM-CD | 75.33% |
| Stream-WAM (Ours) | 75.35% |
Real robot evaluation
We evaluate standard Joint WAM inference, its distilled 1V10A variant, and Stream-WAM on the same real robot manipulation task using a single NVIDIA GeForce RTX 5090 at a 25 Hz control frequency. Stream-WAM reduces Chunk Time to 67.8 ms and completes the rollout in 33 s.
| Method | Inference strategy | Chunk Time ↓ | Episode Time ↓ |
|---|---|---|---|
| Joint WAM | Synchronous | 667.1 ms | 68 s |
| Distilled WAM (1V10A) | One-step distilled | 402.7 ms | 60 s |
| Stream-WAM (Ours) | Action-conditioned streaming | 67.8 ms | 33 s |
Inference efficiency. Task success alone does not characterize runtime efficiency. We therefore report Chunk Time, the latency required to prepare the next action chunk, and Episode Time, the duration of a complete rollout, including inference, execution, and replanning.
Across all three benchmarks, Stream-WAM reduces both runtime measures while maintaining comparable task success. On LIBERO, it achieves a 12.0× Chunk Time speedup over FastWAM and Episode Time speedups of 3.0× and 2.6× on Long and Short tasks, respectively, with 98.20% average success. On RoboTwin 2.0, relative to StarWAM-Joint, Chunk Time falls from 190.17 ms to 47.09 ms and Episode Time from 110.22 s to 77.48 s, while overall success increases from 85.4 to 87.6. On RoboCasa, relative to X-WAM, Stream-WAM achieves a 3.2× Chunk Time speedup and a 1.8× Episode Time speedup, with comparable average success (75.35% versus 75.42%).
| Benchmark | Method | Chunk Time | Episode Time |
|---|---|---|---|
| LIBERO | FastWAM | 493.0 ms | 16.31 s Long / 8.25 s Short |
| LIBERO | FastWAM-Joint-CD | 114.2 ms | 6.89 s Long / 3.74 s Short |
| LIBERO | FastWAM-RTC | 142.3 ms | 6.23 s Long / 3.20 s Short |
| LIBERO | Stream-WAM | 41.0 ms | 5.36 s Long / 3.15 s Short |
| LIBERO | Stream-WAM w/o Action Conditioning | 35.1 ms | 5.20 s Long / 2.92 s Short |
| LIBERO | Stream-WAM w/o Slot Encoder | 36.3 ms | 5.31 s Long / 3.01 s Short |
| RoboTwin 2.0 | StarWAM-Joint | 190.17 ms | 110.22 s |
| RoboTwin 2.0 | StarWAM-CD | 81.21 ms | 102.59 s |
| RoboTwin 2.0 | Stream-WAM | 47.09 ms | 77.48 s |
| RoboCasa | X-WAM | 374.07 ms | 17.36 s |
| RoboCasa | X-WAM-CD | 134.37 ms | 13.04 s |
| RoboCasa | Stream-WAM | 115.98 ms | 9.49 s |
Stream-WAM enables a World Action Model to prepare its next prediction while the robot continues executing the current action chunk.
At its core, the shared actions between adjacent chunks are not treated only as an action continuity constraint. They condition future video generation, allowing the predicted visual trajectory to account for the motion already underway before guiding the next action continuation. One-step consistency distillation and the accelerated runtime further reduce generation latency so that prediction can remain within the available execution window.
Results on LIBERO, RoboCasa, RoboTwin 2.0, and the real robot experiment show that this design reduces Chunk Time and Episode Time while preserving competitive task success. Taken together, the results support a simple conclusion: real-time WAM deployment requires both fast inference and alignment between ongoing execution and predicted world evolution; latency and task success should therefore be evaluated together.
References
- [1]
- [2]
- [3]
- [4]
- [5]
- [6]
- [7]
Citation
@misc{denglab2026streamwam,
title = {Stream-WAM: Streaming Your World-Action Model for Real-Time Robot Manipulation},
author = {{DENG Lab}},
year = {2026},
howpublished = {Project page},
organization = {Shanghai Jiao Tong University},
url = {https://sjtu-deng-lab.github.io/StreamWAM/}
}