llm

2.34× faster LLM decoding: FP8 weights and native speculative decoding on Gemma-4

The full path to 2.34× single-stream decoding of Gemma-4-26B on one A100: a batch-1 byte budget that named the only two levers, FP8 weights, a one-line vLLM patch that switched the model's own speculative head back on, a tuned kernel that did nothing, and the benchmark that first reported the win as a loss. Exact same output.