Google's WikiSkill gives AI agents a persistent memory

What happened

So the agent, aka the model, doesn't truly learn in a continuous sense, which remains an unsolved problem. But it writes better instructions for itself after each run and can pull them up the next time around. It's not elegant, and it's probably more error-prone than real learning, but the study shows it's an effective workaround.

WikiSkill consistently outperforms all previous skill evolution methods in the study. 3 percent. 6 percent on SpreadSheet. The gains vary a lot by task type. Math problems and spreadsheet manipulation see the biggest improvements, while tasks with long document contexts (OfficeQA) show much smaller gains. 5-4B struggle to reliably execute evolved multi-step search strategies across long contexts and fall back to their default behavior.

Why it matters

The work draws on a perspective from Andrej Karpathy about the idea of an "LLM Wiki," which argues for compiling experience into persistent, cumulative knowledge. WikiSkill applies that thinking to automatic skill development for AI agents. WikiSkill organizes an AI agent's workspace into three tiers. The "Raw Layer" stores complete execution traces, from tool calls to results. This data is immutable and serves as raw material.

Above it sits the "Wiki Layer," where raw data gets distilled into structured insights like documented failure patterns and successful strategies. This knowledge layer never resets and only grows with each iteration, according to the researchers. The top tier, the "Skill Layer," holds the active procedural instructions the agent follows when executing tasks. Unlike the wiki, skills can be rolled back if an update hurts performance.

First, an inference agent executes tasks using the current skills and generates execution traces. A "Wiki Maintainer" then analyzes those traces, spots failure patterns and successful strategies, and writes the findings into the wiki. A "Skill Proposer" uses the updated wiki and execution data to suggest targeted skill changes. Finally, a gating mechanism tests the proposed change on a separate validation set to confirm it actually helps.

If it doesn't, the skill gets rolled back, but the wiki stays intact. Even failed proposals aren't lost because the wiki documents what was tried and why it failed, so the Skill Proposer can build on that knowledge in later iterations. The researchers tested WikiSkill across five benchmarks covering math reasoning, web search, spreadsheet manipulation, document question-answering, and interactive tasks in a virtual environment. 5-Flash.

What to watch

Larger models tend to benefit more from evolved skills. But smaller models running WikiSkill can match the performance of larger models that don't use the framework. Skills developed by one model often transfer to another and sometimes even work better than skills the receiving model built on its own. Since that's not always the case, transferability should probably be checked case by case.

Table: WikiSkill (highlighted) achieves the best or statistically equivalent performance across most model-benchmark combinations. Bold values indicate top scores; multiple bold values in the same column are not statistically different from each other. All values are averages across three independent runs.