An AI search for memory designs finds two simple habits that help

Researchers let a coding model rewrite whole memory systems for chat agents. The winners beat published designs on two benchmarks, and one evolved component improved five existing systems when bolted on.

6 min readIIIOpen sourcePreprint, not peer reviewed
Save
A studio still life shows a wooden box with cards, a paper coil, and three torn paper pieces held by a blue paper clip.AI illustration
Illustration: AI generated

What the study shows

Chat assistants increasingly keep a memory of past conversations. Most memory systems are designed by hand. They usually extract facts from chats, store them and retrieve them later. This preprint, which has not been peer reviewed, asks whether an AI can design such systems itself.

The authors built a framework called MAPE. A coding model (the paper names Claude Opus 4.8) writes a complete memory program, starting from one that stores nothing. The program is tested, and a second agent reads the execution logs to explain what went wrong. The coding model then proposes the next version. Each run lasted 20 steps. One mode chased accuracy alone. The other chased accuracy and running cost together.

They tested on two public benchmarks of question answering over long conversations, LoCoMo and LongMemEval. An AI judge marked the answers. The comparison set was seven baselines: feeding in the full history, plain retrieval of similar messages, and five published memory systems, including Mem0 and Zep.

The authors report that every selected evolved program beat the baselines on its own benchmark. On LongMemEval, the best evolved program scored 77% against 66% for the strongest baseline. On LoCoMo with an open Qwen model, the cost-aware program beat the best memory system by about 12 percentage points at about a third of its running cost.

Two patterns recurred. First, the programs kept the raw conversation text as searchable evidence, not only extracted facts. Second, they sorted questions by type and handled each differently. For a date question, code works out the date; for a counting question, code does the counting. Removing raw text from one program cut accuracy by up to about 12 points. Adding the question-routing module to five published systems raised accuracy in all five, by about 3 to 9 points.

The search goal shaped the designs. Accuracy-only search kept adding expensive steps for small gains. Cost-aware search moved work away from model calls when memory is built, towards selective work when a question arrives.

Our reading

In our reading, the most useful result is not the evolved programs themselves. It is the two patterns, and the fact that one of them transferred. A module that helped five independently built systems is a stronger signal than a top score on one benchmark.

A practical example: an assistant is asked "When did I join that support group?" A user once said "last week". Many memory systems store the phrase "last week" and leave the model to resolve it. The evolved cost-aware program converted it to a real date before answering. When the authors swapped in a different answering model, the program that left dates unresolved fell from about 80% to 44% on date questions. The one that resolved them held at about 71%.

The evidence is narrower than the headline numbers suggest. There are two benchmarks, an AI judge with no human check, few search runs and no confidence intervals. The final programs were also chosen by their scores on the same set used to report results.

Free membership

Read the rest of this story

Membership is free. You get every article in full and the Issue every Sunday.

We send a sign-in link, no password needed. Already a member? Sign in

Readiness assessment

III
Open source
LevelStatusEvidence
I  Lab onlyMetThe method was developed and tested in the lab on two research benchmarks, with ablations and transfer tests.
II  BenchmarkedMetResults on two public benchmarks, LoCoMo and LongMemEval-S, compared with seven baselines including Mem0, A-Mem, LightMem, SimpleMem, Zep, RAG and full context, under a shared protocol with two backbones.
III  Open sourceMetThe abstract states code is available at https://github.com/ash-ding/MAPE. I could not check the repository, its licence, or whether it includes the evolved programs and weights.
IV  In productionNot yetNo named organisation is reported using MAPE or its evolved memory programs in production. All evidence comes from benchmarks.

What would move it up
Documented use of an evolved memory program or the composition module by a named organisation in a live product, with real-user results. Short of that, replication by independent groups, evolution runs repeated several times with variance reported, tests on more benchmarks and human-validated grading.

Checked automatically every week, last on 16 November.

The study

Xinheng Ding, Seyyed Saeid Cheshmi, Ziyang Zhang et al.. Discovering Effective Agent Memory Designs through Program Evolution. arXiv, 12 November 2026. Read the paper

Preprint, not peer reviewed · Licence arXiv (no reuse)

Notes

  1. Ding et al., Discovering Effective Agent Memory Designs through Program Evolution. arXiv preprint, 12 November 2026, not peer reviewed. Source
  2. Code released by the authors (contents not checked by us). Source
  3. LoCoMo, GPT-4.1 mini: accuracy-only evolved program 87.94%, cost-aware 85.18% at $2.03 versus $7.21 per 1,000 queries; best baseline Full Context 80.87%, SimpleMem 80.80%.
  4. LoCoMo, Qwen3-30B-A3B: cost-aware program 81.33% at $1.14 per 1,000 queries versus SimpleMem 69.32% at $3.24. Accuracy-only program temporal accuracy fell from 79.84% to 43.80% on the backbone switch; cost-aware held 70.93%.
  5. LongMemEval, Qwen: accuracy-only 77.00%, cost-aware 72.25%, best baseline RAG 66.00%. Ablation: removing raw rounds cut 12.17 points (LoCoMo) and 6.50 (LongMemEval); removing routed composition cut 4.44 and 6.25. Transplant gains 2.75 to 9.25 points across five systems.

Edited and signed by

Stefan PerdalEditor and publisher86 stories edited

Stefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.

How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

More on Agents