Veda-Sparse / Minimax-H3-T2VA-Veda-8NFE-600Step-Preview

Sparse attention, one head at a time.

Attention dominates video-diffusion inference, yet only ~10% of it meaningfully changes the output. Veda trains a lightweight predictor to find that 10% and skip the rest — without touching the backbone weights.

— attention speedup
RTX PRO 6000, 14.4 s
— end-to-end speedup
same clip
— attention kept
keep-ratio
— layers × heads
predictor table

Dense vs Veda

The authors' own clips. Each video is already composited side by side: full attention on the left, Veda on the right, with per-clip diffusion times burned into the title bar. Same prompt, same seed, same Turbo LoRA. This is the honest way to see what sparsity costs.

Loading comparisons…

The plan table

Every cell is one attention head in one transformer block. The colour is the tile shape that head selected. This is the predictor's own learned table, read straight out of the checkpoint's safetensors metadata — not an approximation.

Duration
View
Head × layer hover or focus a cell for detail

layer 0layer 49
head 0head 55

How to read this, and what the numbers mean

Tile shapes read as (temporal, height, width) in tile units. All six cover 128 keys, but they split that budget along different axes — so which one a head picks says something about where that head's important interactions live in the latent grid. Heads converge on different shapes as depth increases, which is how the packer hits its keep-ratio target without running a per-layer search.

The modeled speedup is a model, not a measurement. It is computed in-browser from the plan table: dense cost is ((t·h·w)/128)² per head; sparse cost sums n_tiles × (keep_ratio × n_tiles) over each head's chosen shape. It reproduces the trend — larger gains on longer clips, because dense cost is quadratic in latent tokens while sparse cost is near-linear — but it overstates the real figures (predicts ~7.8× at 5.17 s where the model card measures 4.75×) because it counts arithmetic only, ignoring kernel launch overhead, tile-boundary padding waste, gather cost, and memory traffic. The measured table in the README is authoritative.

plan_mse is regression error against the teacher's own head selection over 400 entries per geometry — lower is a closer fit to the teacher, not a measure of output quality.

Measured performance

From the model card, 10% attention kept. Attention is not the only cost — MLPs, VAE decode, and the audio path remain, so end-to-end lands below the attention figure.

Measured attention and end-to-end speedups by GPU and clip length
GPUClip Attention speedupEnd‑to‑end speedup
RTX PRO 6000 Blackwell16:9 · 14.4 s6.79×3.12×
RTX 409016:9 · 5.17 s4.75×1.77×
RTX 409016:9 · 10.1 s6.22×2.42×
RTX 409016:9 · 14.4 s6.31×2.82×

Run it yourself