AI coding agents can modernize research software but

What happened

The report documents eight case studies, mostly in biology, in which research groups used coding agents such as Codex and Claude Code. The projects range from basic maintenance and targeted optimization to full rewrites in modern programming languages. One of the simpler projects involved modernizing cyvcf2, a Python library for reading genetic data. 5 replaced its outdated build and installation process with a modern one.

The MHCflurry migration was far more involved. MHCflurry is an immunology model that predicts which targets immune cells will recognize. Claude Code and Codex alternated between the roles of developer and reviewer while porting about 10,000 lines of code from TensorFlow to PyTorch. The rustar-aligner project was more ambitious. It rebuilt STAR from scratch in Rust. STAR maps sequencing reads from cells to the corresponding locations in a genome.

The original contains more than 20,000 lines of C and C++ and is no longer actively maintained, even though it remains part of many research pipelines. To check whether the rewrite behaved like the original, the team tested both tools on 10,000 short sequencing reads from yeast cells. 815 percent of cases. 883 percent. The comparison covered more than the mapped location in the genome.

It also included several other key fields that both programs produce for each read. Neither tool mapped any reads that the other failed to map. RustQC delivered the largest speedup by combining 15 separate quality-control tools in a single program. On a large dataset, runtime fell from 15 hours and 34 minutes to 14 minutes and 54 seconds, a speedup of more than 60 times.

Another project, HelixForge, replaced a tool for generating synthetic genomic data with a version that runs on GPUs. 6 times faster than BamSurgeon. 6 times faster. Across the case studies, agents completed well-defined tasks quickly but couldn't reliably judge whether their work was scientifically correct. Even when their code contained errors, the systems often presented it with full confidence.

"With coding agents, it's quite easy to go fast; for now, to go far in science, there's still a need for expert guidance, understanding, taste, and care," writes cyvcf2 developer Brent Pedersen. " He never allowed the models to judge the accuracy of their own work and instead built an independent test harness. The bayesm case study shows how difficult these errors can be to catch.

Its Rust rewrite ran between two and twenty times faster than the original, but the first versions of two advanced methods contained errors that were hard to spot from the output alone. In one method, the agent inverted a key control parameter, causing the program to use the reciprocal of the intended values. A separate bug affected the calculation itself.

Researchers only found it after running a detailed calibration test against thousands of synthetic datasets with known results. Another method, called HART, produced plausible results overall but still contained several flaws. These included needlessly expensive calculations and an incorrectly scaled correction factor. Plausible test results alone couldn't establish that the code was correct. An earlier attempt to port MHCflurry to PyTorch had failed in early 2025.

Hifiasm assembles a complete genome from many short fragments. 5 to optimize it, the researcher built a test setup with separate training and validation datasets. The model then found changes that cut runtime on real human genome data by nearly 15 percent. SIM, a library for simulating genetic data, required even less human involvement. 2 found ways to optimize individual parts of the program in a single pass.

Why it matters

Developer Sergey Feldman now attributes the failure to the models available at the time rather than the coding tools themselves. In his view, only newer model generations became reliable enough to handle much of this work on their own. The projects followed a consistent division of labor. Humans defined the goals, success criteria, and validation methods, while agents handled implementation. The hifiasm project shows how this worked in practice.

What to watch

A second pass with a newer model found more improvements. Together, the changes reduced runtime by about 31 percent without changing the output. The authors also provide rough estimates of the potential savings. If agents could resolve between one-quarter and one-half of all installation problems affecting research software, the saved research time across 100 packages would be worth between $600,000 and nearly $5 million.

For NumPy alone, the report estimates that agents could save about 650 hours of maintenance work each year. Long-term maintenance remains a major open problem alongside validation and scientific accuracy. Low-cost rewrites could fragment user communities and spread experienced maintainers' already limited time even thinner. The teams took different approaches to ownership and maintenance. Some changes went directly into the original projects.