ICLR 2027 submission · under review

VFlash-FM

Flow-Trained One-Pass Drafting for Video Language Models

Demo How it works Results

VFlash-FM speeds up video language models by proposing a whole block of 16 tokens in one drafter forward, which the unchanged target verifies in one pass. The drafter never processes the video: it reads the target's hidden states, and it is trained along a discrete flow from masks to the target's continuation instead of only at the all-mask start. On LLaVA-OneVision-7B this gives a 3.85× macro-average decoding speedup across five video benchmarks.

Side by side: autoregressive decoding vs VFlash-FM

Four clips, each decoded twice on the same H100 and played back at a quarter of real time. The clip plays above its prompt while both decoders write. In all four the VFlash-FM output is identical to the autoregressive output, token for token, so the panels differ only in how fast they fill.

On a small screen, use the player’s full-screen button to read the text inside each clip.

Decode time after prefill, recorded with the paper's evaluation harness: native grouped-query attention, preallocated KV caches, block 16, one drafter forward per block, greedy decoding. Autoregressive decoding measures 10.6 to 11.1 ms per token on these clips.

How VFlash-FM works

The drafter never processes the video. It reads the hidden states the frozen target has already computed for every context position, proposes a whole block of 15 tokens in one forward, and the unchanged target verifies the block in one forward. Whatever is accepted is exactly what the target would have generated on its own.

Autoregressiveone target forward per token target 7B less target 7B top target 7B and target 7B black target 7B pants target 7B , target 7B stands target 7B in 8 tokens = 8 target forwards, each over the full 25,119-token context (10.7 ms each on the demo clip) VFlash-FMone drafter forward + one target forward per block of 16 video +prompt prefill once frozen target 7Bhidden states cached hidden states linearprojector keys & values drafter4 layers block in: anchor + 15 masks sleeve M M M M M M M M M M M M M M M proposedone drafter forward one forward sleeve less top and black pants , stands stands standing the el horse horse , , frozen target 7Bverifies in one forward verifiedone target forward sleeve less top and black pants , stands stands standing the el horse horse , , → in committed8 tokens this cycle accepted prefix + the target's correction; the rest is discarded sleeve less top and black pants , stands in the next cycle starts from “in” as its anchor
Autoregressive one target forward per token target 7B less target 7B top target 7B and target 7B black target 7B pants target 7B , target 7B stands target 7B in 8 tokens = 8 target forwards, each over the full 25,119-token context (10.7 ms each on the demo clip) VFlash-FM one drafter forward + one target forward per block of 16 frozen target 7B video + prompt prefilled once linear projector: keys & values drafter 4 layers one forward: anchor + 15 masks in 15 tokens out proposed one drafter forward sleeve less top and black pants , stands stands standing the el horse horse , , target verifies all 16 in one forward verified one target forward sleeve less top and black pants , stands stands standing the el horse horse , , target’s correction → in accepted prefix + target’s correction; the rest is discarded committed 8 tokens this cycle sleeve less top and black pants , stands in the next cycle starts from “in” as its anchor
One real cycle from a VideoDetailCaption clip. The drafter reads the target's cached hidden states through a linear projector and fills a block of masks in one forward. The target then scores all 16 tokens in one forward: the prefix that matches its own greedy choice is accepted (green), it supplies the token at the first disagreement (orange), and the rest is discarded. Here 7 drafts were accepted, so 2 forwards committed 8 tokens where autoregressive decoding needs 8 target forwards.

Training is what gives the drafter its early-block accuracy. The masked block is treated as the state of a discrete flow that starts from all masks and ends at the target's continuation, and the same network is also supervised at partially revealed blocks that its own reveal rounds produce, with revealed tokens that may be wrong, while the loss always points at the target's continuation. It learns to keep what it was right about, overwrite what it was wrong about, and predict the rest. Deployment stays a single forward from masks, so the training adds nothing to decode time.

Decoding speedup over autoregressive decoding

LLaVA-OneVision-7B, five video benchmarks

Fastest listed variant of each method. Baselines are the published numbers of each study on its own hardware and protocol; ours are 50 clips × 3 seeds per benchmark on one H100, greedy, block 16, one drafter forward per block.

Values as a table
MethodVDCVideo-MMEMVBenchMLVULVBMacro avg.
SpecVLM (0.5B drafter)1.831.821.771.761.861.81
ParallelVLM (0.5B drafter)2.122.062.152.142.102.11
SPARROW (learned drafter)2.842.962.69not reported2.772.82
VFlash-FM (ours), Tree-603.6444.1763.6064.0873.7343.850

VDC: VideoDetailCaption; LVB: LongVideoBench. With chain verification VFlash-FM reaches 3.773× macro on the same benchmarks.