DENG Lab · Shanghai Jiao Tong University

Wave ForcingTowards Speed-of-Light Streaming Video Generation

Yijie Jin1*, Yuxuan Liu1*, Jiajun Xu1,2*, Ye Huang3*, Chuzhi Wang1, Jiayang Li4, Ruiyang Chen1, Chenling Hong5, Xu Wang1, Zhijie Deng1†

1DENG Lab, Shanghai Jiao Tong University 2Xi'an Jiaotong University 3Peking University 4East China Normal University 5National Tsing Hua University * Equal contribution  ·  † Corresponding author

The page guide: a chibi whale-maid with long blue hair

Hi! I'll guide you through the page. The short version: once the pipeline fills, every forward round outputs one chunk.

Wave Forcing in 90 seconds.

TL;DR

Multi-step streaming video, one chunk per forward round.

Wave Forcing (WF) co-designs generation, training and execution. A causal wavefront lets newer, noisier chunks read earlier, cleaner ones, so denoising stages run concurrently on full-model replicas. Wave Runtime (WaveRT) hides K/V delivery behind compute and pipelines VAE decoding.

147.6DiT FPS 4-step WF on 8×H200 134.8 FPS end to end with VAE decoding; BF16/FA3, 832×480.‡
7.07× single-GPU E2E throughput vs the 19.06 FPS single-GPU baseline.‡
2.53× over the faster CP baseline WP vs Ulysses / Ring, DiT-only, Sage+W8A8, 1.3B.‡
94.44 Dynamic Degree (vs 51.39) 5-step WF vs Rolling Forcing; VBench Total 81.64 vs 81.15.

‡ The 1.3B rows use the released 4-step and 5-step checkpoints. Timings for scales above 1.3B use randomly initialized (dummy) DiT weights with a real-weight VAE, because pretrained weights at those scales are unavailable and the 14B scale could not be adequately trained. FPS is 1593 decoded frames divided by the median of three runs.

Abstract

Real-time long-video generation requires continuous output with consistent appearance and natural motion. Existing chunk-wise autoregressive methods face challenges in both generation efficiency and long-video quality: serialized denoising and history-cache refresh require two forward passes even with one-step denoising, and longer rollouts can degrade quality and cause camera drift. We propose Wave Forcing (WF), which co-designs generation, training, and execution for continuous video output with multi-step denoising. WF uses chunk-wise causal attention to access earlier, cleaner chunks before their denoising is complete, and distributes staggered denoising and cache-refresh stages across full-model replicas to achieve 1F1C: one output chunk per concurrent forward round after pipeline fill. To adapt the model to this continuous generation process, we develop a three-stage training procedure that successively introduces mixed-noise denoising, motion supervision, and causal adaptation using self-generated history. To execute this process efficiently, Wave Runtime (WaveRT) uses Wave Parallelism (WP) for early K/V delivery and communication–computation overlap, together with optimized kernels and pipelined VAE decoding.

Experiments on Wan2.1-T2V-1.3B show that 5-step WF maintains overall quality comparable to Rolling Forcing while achieving higher Dynamic Degree on both short and long videos. Our system efficiency experiments show that 4-step WF achieves 147.6 DiT FPS and 134.8 end-to-end FPS on eight H200 GPUs, yielding 7.07× the single-GPU end-to-end throughput. In DiT-only tests with matched SageAttention+W8A8 backends, denoising steps, and GPU counts, WP delivers up to 2.53× the throughput of the faster of the Ulysses and Ring context-parallel baselines across six model scales (1.3B–14B).

Method · Wave Forcing

A bounded-width causal wavefront.

Earlier chunks run later denoising stages while newer chunks run earlier ones. Each denoising stage and the store pass own a full-model replica, so the staggered tasks advance together as a moving wavefront.

Sequential · NF1C

N forward rounds per chunk

Chunk-wise autoregressive methods such as Self Forcing and Causal Forcing denoise each chunk on top of earlier content, then run one more forward pass, the store, to refresh the K/V history from the clean result. Denoising and store sit between consecutive outputs, so even one-step denoising needs two forward passes per chunk.

Wave Forcing · 1F1C

One chunk per concurrent forward round

WF keeps every denoising update but staggers them: with S denoising stages plus store on S + 1 replicas, one completed chunk leaves the pipeline per round after fill. Chunk-wise causal attention keeps full attention within a chunk; across time, query slot s reads K/V slot j only if j ≥ s, which means itself and all cleaner predecessors, never a later chunk.

Wavefront explorer S = 4

round k = 0
denoising forward this round store (K/V refresh) next round chunk out
Forwards running this round1
Chunks out0

Chunk-wise causal mask M(s, j) = 1[j ≥ s]

Rows: query slot s (chunk i = k − s). Columns: K/V slot j. Every slot also reads the anchor, clean history and live store.

Complete wavefront, causal K/V delivery and continuous chunk output(a) Complete wavefront4 denoising stages + storeStage sChunk i →0123456s=0s=1s=2s=3store (4)Current i+s=5Next i+s=6Latent advances one stage next roundFillk=0k=1k=2k=3k=4Bounded width ≤ S+1 = 5, including store(b) Causal K/V deliveryOne full-model replica per stagestorei=1s=3i=2s=2i=3s=1i=4s=0i=5Cleaner sourcesConsumerNot readShorter context can create a lead.s=2s=1PComputeEarlier workPAttn + restK/VReady before useP: QKV + RoPE; delivery overlaps work.Illustrative edge; other inputs ready.(c) Continuous outputSame 5 forwards per chunkForward rounds12345SF / 1 replica5F1C----chunk01 chunk / 5 roundsWF / 5 replicas1F1Cchunk0chunk1chunk2chunk3chunk45 chunks / 5 roundsWF after fill; rounds ≠ wall time.

scroll sideways →

Causal wavefront and continuous output (S = 4). (a) Denoising and store advance on anti-diagonals, with dashed cells marking the next round. (b) One later-layer K/V edge arrives before use. (c) Both schedules retain four denoising forwards plus store per chunk; over five rounds, one SF replica emits one chunk, whereas five WF replicas emit five after fill. Wall-clock gains depend on stage costs and overlap.

Method · Training

Training for the deployment graph.

The causal mask removes access to later, noisier chunks and changes the conditioning Rolling Forcing learned. Three stages adapt Wan2.1-T2V-1.3B to it.

1

Mixed-noise initialization

RF's mixed-noise joint-denoising setup, with bidirectional attention across chunks at different noise levels. Each step samples an RF or an SF rollout with equal probability and trains with DMD. SF rollouts help stabilize motion.

DMDRF | SF rollout
2

Off-policy DMD

Following LPM 1.0: consecutive teacher chunks are re-noised at staggered timesteps, with earlier teacher chunks as clean K/V history. DMD and LPIPS supervise the same prediction, adding stronger motion and subject guidance.

DMD + LPIPSλₚ = 0.25
3

Causal adaptation, self-generated history

Each iteration uniformly samples a self-generated RF, WF or SF rollout for on-policy DMD. The WF branch uses the causal context, including store-refreshed history from its own outputs.

on-policy DMD⅓ RF + ⅓ WF + ⅓ SF

The 4-step student starts from SF's ODE initialization. The 5-step student reuses the released RF weights and skips Stage 1.

System · Wave Runtime

Wave Parallelism hides K/V delivery behind compute.

WF exposes stage concurrency, but K/V dependencies can still stall attention. WaveRT, built on SGLang Diffusion's model implementation, publishes K/V early from stage-owned replicas and overlaps delivery with computation when cleaner producers run ahead.

Stage-owned replicas

Each denoising stage and the store own a full-model GPU replica. Rank s updates chunk k − s and hands it to rank s + 1 for the next round.

Early publication

Cleaner ranks read fewer keys, so they can run ahead. K/V is published after QKV + RoPE, and readiness is checked just before attention.

we = max(0, τe − Δe)

Paged delivery

K/V is written directly into ordered attention slots and signalled ready before use, with no remote concatenation. Attention consumes the same K/V values as synchronized execution.

Kernels + pipelined VAE

SageAttention and W8A8 speed up local operators. The VAE decoder is split into profile-balanced contiguous stages on dedicated GPUs, overlapping DiT.

Wave Parallelism · inside one round

K/V ships while compute runs

scroll sideways →

Bars are measured tick p50 for 1.3B BF16 (4-step, 8 GPUs); each one adds a mechanism to the one above, and Early publish is the Overlap column. Block widths in the diagram are schematic.‡

One round, two GPUs, three layers. GPU 1's noisier chunk attends to K/V from GPU 2's cleaner chunk. Δ is the time from publication to use and τ the delivery cost, so the exposed wait is w = max(0, τ − Δ).
Paper figure: K/V delivery schedules
Matched synchronized and early-publication schedules inside one forward roundK/V delivery across five layersTwo ranks; matched costs; other inputs ready.(a) Layer-synchronized exchangeProducerK/V transferConsumerPC1PC11PC2PC22PC3PC33PC4PC44PC5PC5501234567(b) Early publication + point-of-use waitingProducerK/V transferConsumerPC1PC11PC2PC22PC3PC33PC4PC44PC5PC5501234567Exposed waitBarrier idleDashed: ready before useP: QKV + RoPE; C: attention + remaining block(c) Submit, publish, consumeSubmit active chunks / roundQKV + RoPEWait QKVeventLocal computeIPC K/V copyReceiver: final attention slotsAHist.StoreSelfCleanerK/VA: anchor; local prefix/selfcopies remainPublishtick flagWait all inputsAttention reads ordered slotsNo remote K/V concatenationFinish layer → next layerRound end: drain outgoing K/Vthen latent All-Gather / slot reuse

scroll sideways →

Hiding K/V delivery with identical schematic operation costs. (a) Layer barriers expose communication. (b) Early publication hides later waits, but not dependencies or the initial wait. (c) Direct writes to receiver slots precede readiness signals and consumption.

VAE pipeline parallelism

Three decode stages, each about one round long

scroll sideways →

8×H200: four GPUs denoise, one runs the store pass, three decode. The round and the stage times are measured; the two shorter FLOP-split stages are not reported, so their widths are schematic. End to end the runtime reaches 134.8 FPS against 147.6 DiT FPS, and the first output still pays the pipeline-fill latency.‡

Eight DiT rounds of 78.88 ms. The top row is the DiT (four denoising GPUs and the store GPU), each VAE row is one GPU, and tiles in the out row are decoded chunks leaving the pipeline. The strip at the bottom shows where the decoder is cut into stages, next to one round for scale.
Paper figure: streaming VAE schedules
(a) Blocking single-GPU VAE1 VAE GPU; separate DiT/VAE resourcesDiT+storeVAE0wait001wait112wait22DecodedTimeNext round waits for decode.(b) Async single-GPU VAE1 VAE GPU; separate DiT/VAE resourcesDiT+storeVAE000111222DecodedQueueTimeDiT advances; VAE queues chunks.(c) Unbalanced VAE PP3 VAE GPUs; latent-frame microbatchesF1F2F3F4111F1222F2333F3444F4VAE 1VAE 2VAE 3DecodedTimeSlow middle stage grows the queue.(d) Load-balanced VAE PP3 VAE GPUs; latent-frame microbatchesF1F2F3F4111F1222F2333F3444F4VAE 1VAE 2VAE 3DecodedTimeBalanced stages sustain arrivals.(e) Profile-guided splitConstruct the stages in (d)Ordered decoder units12345678Profile each unit in streaming mode12345678Unit times shown schematicallyContiguous min-max splitStage groups (illustrative boundaries)VAE 1123VAE 245VAE 3678Equal width = balanced stage timeDiT + storeTracked microbatch (F1)Wait / queueVAE: longer → darker

scroll sideways →

Streaming VAE schedules. (a, b) Blocking vs asynchronous decoding with matched costs. (c, d) Unbalanced vs load-balanced pipeline parallelism under identical workloads and arrivals; darker teal marks longer stage service times. (e) Profile-guided contiguous decoder partitions. The first output still pays pipeline-fill latency.

Results · Efficiency

147.6 DiT FPS, 134.8 end to end.

4-step WaveRT at 832×480 on one 8×H200 node: four GPUs denoise, one runs store, and three decode as a load-balanced VAE pipeline.

147.6DiT FPS 1.3B, 4-step, BF16/FA3 Diffusion, communication, store and fill/drain.
134.8E2E FPS same runs, through VAE completion 127.06 FPS with synchronized K/V on the same graph.
7.07× vs one GPU Single-GPU rolling inference: 19.06 E2E FPS.
33.50FPS 14B, Sage+W8A8 6.08× its 5.51 FPS single-GPU reference.

The 1.3B rows use the released 4-step and 5-step checkpoints. The 14B rows use randomly initialized (dummy) DiT weights, because pretrained weights are unavailable at that scale and there was not enough compute to adequately train it; the VAE uses real weights. FPS is 1593 decoded frames divided by the median elapsed time over three runs; loading and file encoding are excluded.

ConfigurationGPUsTick p50 (ms)Steady (FPS)DiT (FPS)E2E (FPS)Speedup
1.3B, 4-step
Baseline1––33.8619.061.00×
WaveRT (Sync)891.96130.50129.31127.066.67×
WaveRT878.88152.13147.58134.817.07×
WaveRT + Sage878.36153.15147.78134.797.07×
WaveRT + Sage + W8A8876.05157.79145.47132.896.97×
1.3B, 5-step
Baseline1––29.1217.461.00×
WaveRT8120.5399.56104.8296.465.52×
WaveRT + Sage8120.5299.57104.8096.485.53×
WaveRT + Sage + W8A88120.5999.51103.4995.155.45×
14B, 4-step†
Baseline1––6.315.511.00×
WaveRT (Sync)8464.2725.8525.5625.474.62×
WaveRT8413.8029.0028.7428.645.20×
WaveRT + Sage8414.0328.9828.7228.625.19×
WaveRT + Sage + W8A88354.4533.8633.6433.506.08×
Runtime throughput. WaveRT uses the WF graph and paged K/V unless marked Sync; Baseline is single-GPU rolling inference. BF16 with FA3 unless a kernel suffix is given. GPU counts include denoising, store and VAE: 4+1+3 for 4-step, 5+1+2 for 5-step. Speedup is E2E relative to the same-scale, same-step baseline. † Random-initialized DiT weights with a real VAE: system scaling only, not a quality result. Teal marks the best value per model/step group.

Wave Parallelism vs context parallelism

DiT-only throughput (dummy weights, VAE disabled) against attention-level context parallelism, Ulysses and Ring, across six model scales. Steps and GPUs are matched within each row: 5 steps on 6 GPUs for 1.3B–9B, 4 steps on 5 GPUs for 14B. Under Sage+W8A8, WP comes first at every scale, at 1.025–2.530× the faster CP. Under BF16 it leads at five of six scales (0.985–1.862×); at 9B it trails Ulysses by 1.53%.

DiT throughput (FPS) by model scale

Sage+W8A8 · higher is better

† supplemental measurements with a target-stopped repetition protocol (see the paper's appendix).

K/V transport ablation

Same model, WF graph, schedule, topology and local compute per row; only the execution strategy changes. Sync gathers layer K/V before attention; Overlap publishes it early; One-sided uses CUDA IPC P2P; Staggered prefetches readiness checks; Paged writes directly into final attention-buffer slots. Sync → Paged lowers BF16 tick p50 by 14.2% at 1.3B and 10.9% at 14B.

BackboneComputeSyncOverlapOne-sidedStaggeredPaged
1.3BBF1691.95685.73081.66981.95378.882
1.3BSage91.44184.10380.93481.06278.356
1.3BSage+W8A888.71481.59578.03078.08176.052
14BBF16464.270449.689425.907425.188413.795
14BSage467.526439.988426.215425.601414.029
14BSage+W8A8407.215379.576365.922366.697354.447
Pooled steady tick p50 (ms, lower is better) for the 4-step, 8-GPU (4+1+3) setup across three fixed repeats. Dummy DiT weights, real-weight VAE.

Results · Quality

Quality holds, and the video moves.

5-step WF matches Rolling Forcing's overall quality while lifting Dynamic Degree on both short and 100-second videos, with no unusable score on any metric.

81.64VBench Totalvs 81.15 for Rolling Forcing.
94.44Dynamic Degree, short videosvs 51.39 for Rolling Forcing.
91.45Dynamic Degree, 100 s videosVBench-Long; vs 36.63 for Rolling Forcing.
97.10Subject Preservationvs 92.75; Camera Drift stays low at 0.37 (RF: 0.17).
Standard VBench ↑VBench-Long ↑StabilityFPS ↑
MethodStepsGPUsTotalQualitySemanticDynamicQualityDynamicSubj. ↑Drift ↓DiTE2E
Rolling Forcing5181.1583.7770.6651.3981.1836.6392.750.1729.1217.46
+ Causal Attention5880.8983.3571.0537.5080.5046.6091.260.40104.8596.56
Causal Forcing (Full Attn.)4179.6182.8366.72100.0082.1596.6171.1111.1433.8819.06
Wave Forcing (Ours)4880.7483.9367.9584.7280.8974.9791.880.36147.58134.81
+ SageAttention881.0384.0269.0765.2881.1677.2490.790.36147.78134.79
Wave Forcing (Ours)5881.6484.4070.5994.4481.2691.4597.100.37104.8296.46
+ SageAttention881.0283.9269.4180.5681.1491.1494.430.37104.8096.48
Short- and long-video comparison on Wan2.1-T2V-1.3B-scale models. Dynamic is Dynamic Degree; Subj./Drift are Subject Preservation/Camera Drift (0–100). Short videos: one 129-frame video per VBench prompt. Long videos: 100-second videos on 200 MovieGen prompts. All at 832×480, 16 FPS, seed 42. Teal marks the best value per metric; coral marks unusable outcomes.

Training ablations

4-step models on long videos. Skipping off-policy DMD collapses motion (Dynamic Degree 15.73) and loses the subject. Removing RF raises motion but destabilizes framing (Drift 6.24). Removing WF gives the steadiest camera but halves motion (48.12). Removing SF degrades quality. The full recipe is the only one with no unusable score.

ModelQuality ↑Dynamic ↑Subj. ↑Drift ↓
w/o off-policy DMD76.6815.7376.530.26
w/o RF80.1988.8277.486.24
w/o WF80.3448.1295.640.10
w/o SF75.3523.6684.041.89
Ours full80.8974.9791.880.36
Quality/Dynamic are VBench-Long scores. All rows evaluate final models.

Qualitative

Long rollouts that keep their subject.

One 60-second take

A single 5-step Wave Forcing rollout, played back unedited. Nothing is cut or stitched: every chunk is generated causally from the ones before it.

The page guide, seen from behind, watching the video

I'll just watch this one for a minute…

Frames at 0 to 95 seconds for Causal Forcing, Rolling Forcing and 4-step and 5-step Wave Forcing on a woman reading a book
Qualitative comparison. Rows: Causal Forcing, Rolling Forcing, and 4-step and 5-step Wave Forcing. Causal Forcing loses the subject at 30 s and 75 s, leaving pillows and the headboard in view; Wave Forcing keeps the reader while her pose and interaction with the book change.
Frames at 0 to 95 seconds for the full 4-step model and each training ablation on a boy eating a burger
Qualitative training ablations. The full 4-step model against each ablation: without off-policy DMD the subject moves out of view; without SF the frames degrade.

Limitations

What's still open.

The VAE output path

Pipelining overlaps and partitions decoding but doesn't remove its finite capacity. The DiT/E2E gap (147.58 vs 134.81 FPS at 4 steps) is consistent with residual decoder and output-path limits; matched VAE-on/off runs are needed to isolate the bottleneck.

Training complexity

Simplifying mixed-noise initialization, teacher guidance and causal adaptation to self-generated history, while keeping motion and long-video stability, remains open.

Scalability

Pipeline width is bounded by the number of denoising steps, which limits how far one pipeline scales to more GPUs. WP's advantage over CP also narrows for larger models.

Citation

BibTeX

Read our arXiv paper and cite it using the BibTeX below.

@misc{denglab2026waveforcing,
  title        = {Wave Forcing: Towards Speed-of-Light Streaming Video Generation},
  author       = {Jin, Yijie and Liu, Yuxuan and Xu, Jiajun and Huang, Ye and Wang, Chuzhi and
                  Li, Jiayang and Chen, Ruiyang and Hong, Chenling and Wang, Xu and Deng, Zhijie},
  year         = {2026},
  eprint       = {2611.00859},
  archivePrefix = {arXiv},
  url          = {https://arxiv.org/abs/2611.00859}
}