the speed wasn't real

Context this is a follow-up to WFD 37, where i wrote about diffusion LLMs generating text in parallel instead of one token at a time, how that changes every prompting technique you've internalized, and how the speed feels like a genuinely different category of fast. after more experimentation, the conclusions in that piece didn't hold up.

the selling point

the speed was the entire selling point. sub-200ms time-to-first-token, over a thousand tokens per second, the whole output materializing at once instead of streaming in. that's a real measurement from a real model, and experiencing it for the first time genuinely recalibrated what i thought inference could feel like. but after spending a while experimenting with diffusion models at output quality comparable to GPT 5.2, the advantage either turned to shit or landed somewhere so negligible it stopped being worth talking about. the gap that felt revolutionary when i wrote WFD 37 doesn't survive the quality bar that actually matters.

what actually happened

the practical experience was worse than the benchmarks suggested. i was calling the API through OpenRouter at around 20 requests per second and kept hitting model errors, not rate limits, actual inference failures that killed the workflow. this is probably fixable by routing through the provider directly instead of through OpenRouter, but when the whole pitch is speed and the reality at volume is a stream of broken responses, the motivation to debug evaporates. prompt adherence was also slipping even after i restructured everything along WFD 37's guidelines: positive framing, inline constraints, template-driven output. the model still drifted, and at that call frequency the drift compounds fast.

the prompting still holds

the prompting insights from WFD 37 still hold in theory. show-don't-tell, positive instructions over negation, inline constraints, all of that still tracks for how diffusion models process prompts. the frame is what fell apart. the entire piece was built around "these are fast in ways that feel wrong," and the honest answer after more testing is they're fast when quality isn't the priority. push them to produce reasoning and coherence at frontier autoregressive quality and the denoising steps pile up, the parallelism advantage erodes, and what you're left with is a different architecture that arrives at roughly the same speed for roughly the same quality.

benchmarks measure the easy stuff

benchmarks measured what was easy to measure. "1100+ tokens per second" is real, but the quality of those tokens at that speed falls well below what i actually need, and when the quality goes up the speed comes down. at the tier where diffusion output is actually usable for real work, the difference between it and a good autoregressive model is a rounding error.

the selling point

what actually happened

the prompting still holds

benchmarks measure the easy stuff

33

Atlas: structural code-intelligence for LLM agents (an empirical evaluation)