LFM2.5-Encoders for Fast Long-Context Inference on CPU

What happened

5-Encoders come from the same family but serve a broader purpose. They're pre-trained with a masked-language objective, so you can fine-tune them for classification, token-level tasks, and search alike. Search is just one thing an encoder enables. That's why we built a general-purpose model instead of reusing the retrievers. Encoders power many modern production NLP applications: classifiers, intent routers, safety filters.

These jobs run all day, usually on CPU, on ever-longer inputs. BERT established this class of model, and recently ModernBERT pushed its accuracy, speed, and context further. 5-Encoders take the next step on the LFM2 architecture, where cost grows slowly as inputs grow. 5-350M.

Then we turn each causal decoder into a bidirectional encoder with a few changes: We fine-tune each model fully on every task and report the resulting score. Across the table, that's 14 models on 17 tasks pulled from GLUE, SuperGLUE, and multilingual classification. We report the mean across five held-out seeds, so the numbers are stable run to run. The full framework and raw results are open-sourced.

5-Encoder-350M ranks fourth of the 14 models. 5B model nearly 10 times its size. 5-Encoder-230M beats ModernBERT-base and every EuroBERT model, while being smaller than most of them. 5-Retrievers here. Our encoders inherit the LFM2 backbone's fast inference. Since both our encoders and ModernBERT support an 8,192-token context, we measure speed across the full range. Our encoders show their biggest edge on CPU.

5-Encoder-230M is the fastest at every sequence length (even faster than the smaller ModernBERT-base for short inputs). 5-Encoders rise into the mid-range before tapering. 5-Encoder-230M. 7x faster. For developers, that means you can scan or classify a full contract, transcript, or long support thread in under 30 seconds on a laptop CPU.

On GPU, a similar pattern holds with a smaller margin: ModernBERT-base leads below ~1K tokens on the Apple GPU. Our encoders take the lead from about 2K tokens. 5-Encoders are the faster choice, and if you're running on CPU, dramatically so. 5-Encoders. 5-Encoder when you have a high-volume understanding task, such as classification, routing, extraction, or scoring, that runs constantly and has to stay cheap and fast.

Why it matters

5-Encoder-350M. They match the quality of larger models but stay fast as inputs get longer. This means you can run document-scale jobs on the hardware you already have, even on CPU. With these, you can build intent routers, policy linters, PII detectors, and text classifiers that run cheaply, all day. See the live demos below. 5-Retrievers, built for multilingual search.

What to watch

For jobs like these, a fine-tuned encoder is smaller, faster, and far cheaper to run than a generative LLM, and it fits on the CPUs you already have. You can start in a few lines. Load a model with transformers. Then run it directly for masked-token prediction, or attach your own head and fine-tune it for your task.

For downstream tasks, load the encoder body and attach your own head (classification, token classification, regression, retrieval): If your GPU supports it, use Flash Attention 2 for the highest efficiency: A base encoder gives you general-purpose representations, not task outputs. So you fine-tune it for each task. Our fine-tuning tutorial walks through fine-tuning on long legal documents with an 8k context. We can't wait to see what you build. If you use this work, please cite the release blog: