Up to 3.2x Faster Inference with LFM2.5-DSpark

What happened

5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality: The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation.

6B, speedup on the MacBook is especially noticeable, as it pushes the interactivity level a user can enjoy far beyond the throughput offered by most proprietary cloud models (around ~140 tok/s, depending on the dataset). 6B. 2B-Instruct, we see much more variance in dataset acceptance rates, so speedup varies by as much as 52% depending on the underlying text distribution.

5-8B-A1B, the acceptance rate increases compared to two dense models, yet on-device we get only an 18% improvement on average. cpp's Metal backend, and to the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step. Running the DSpark draft models with SGLang requires an SGLang build with DSpark support for LFM2 targets (PR #31041).

Why it matters

Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify.

Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3, DFlash, and, most recently, DSpark, which combines three components: We follow the DSpark recipe with a larger and more diverse data mix covering SFT, chat, code, and function-calling data. Based on our ablations, the first versions of the draft models are simplified attention-only draft models, with 5 layers and a block of 9.

For each draft model, we ran 15 epochs on the entire dataset and selected the epoch with the highest acceptance rate rather than the lowest loss. The resulting draft models are relatively small, with each around ~300M parameters. Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place.

The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is unchanged. cpp (implementation builds on top of the official codebase, which we run with experimental metal kernels) and SGLang ( implementation builds on the official SGLang implementation of DSpark). cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens.

We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Both configurations use a DSpark block size of 9, a batch size of 1, and a temperature of 0. We evaluate them on five benchmark datasets. All three drafter models deliver noticeable throughput improvements on both the large-scale accelerator (H100) and the edge deployment (M4 Max MacBook).

What to watch

Launch the target with the draft attached: Then query the OpenAI-compatible endpoint at http://localhost:30000/v1. json; the baseline is the same command without the three --speculative-* flags. cpp build (PR#27383). The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact: the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted.

We can’t wait to see what you build. 2x Faster Inference from H100 to MacBook", Liquid AI Blog, Aug 2026.