Newer Models, Same Advantage

What happened

Three months ago, we published a paper on DharmaOCR and open-sourced one of the models. The objective was specific: optical character recognition engineered for Brazilian Portuguese. The training pipeline was built in two stages. The first was a supervised fine-tuning step, drawing on a broad collection of Portuguese-language files from different sources, formats, and levels of complexity.

This distinction is where specialization becomes a structural question rather than a design preference. When a model is trained on a restricted domain — a single language, a bounded document type, a specific task — All of its parameters are dedicated to that specific task.

When a model is trained to cover a broader range of domains — a multilingual model handling N languages, for instance — those same parameters must be distributed across all of them. The distribution is not linear: the neuron superposition principle means individual parameters can encode multiple features simultaneously. But the division is real, and its consequences are real.

A model covering more ground commits less to any given part of it. DharmaOCR was trained to accept that constraint in reverse. The model is not designed to be the best option for other languages, and was never intended to be.

Why it matters

This stage aligned the model's weights to the specific vocabulary, syntax, and document structures of Brazilian Portuguese — concentrating representational capacity on the target language rather than distributing it across a broader multilingual space. The second stage applied Direct Preference Optimization: rather than training only on correct transcriptions, the model learned from comparative preference data between competing outputs, teaching it to consistently select the better extraction at inference time.

This stage addressed a different problem: not accuracy, but stability. By suppressing the failure modes that cause generative models to produce repetitive or incoherent output, DPO reduced inference time and cost, and materially improved the reliability of what the model delivered in production. The combined result was a model that achieved the highest extraction quality score with the lowest degeneration rate on a Portuguese-focused benchmark. Both stages were necessary.

The fine-tuning stage built domain competency; the DPO stage ensured that competency held under the conditions where models tend to fail. OCR models have been moving quickly. But the gaps that originally motivated DharmaOCR's design (in extraction quality on complex documents and in model stability under production conditions) have not closed. They have, if anything, become more instructive as the field has changed.

The proliferation of multimodal generative models made language model-based OCR widely accessible, and the wave of fine-tuned OCR variants that followed reflects how fast that adoption has moved. That proliferation has not, however, changed the fundamental character of the technology. Every OCR system built on a generative model is probabilistic. Transcription errors are an inherent variable of this probabilistic technology.

What differentiates models is how many errors they make and of what kind. That is determined by two things: the structure of the model (its architecture and parameter count) and how those parameters were trained for the task. Architecture and parameter count establish the ceiling on what a model can learn. Training determines how that capacity is allocated.

What to watch

In exchange, every parameter available to the network could be oriented toward the specific vocabulary, morphology, and orthographic patterns of Brazilian Portuguese — the most directed possible use of the model's resources for that domain. That concentration is the structural basis of an inherent advantage over multilingual and broader-domain models.

The advantage does not depend on having a larger architecture or a more sophisticated training procedure than competitors use — new architectures and new training techniqu