# An AI search for memory designs finds two simple habits that help

Researchers let a coding model rewrite whole memory systems for chat agents. The winners beat published designs on two benchmarks, and one evolved component improved five existing systems when bolted on.

- Category: Agents
- Readiness level: III (Open source) on Science Underway's four-step scale, as proposed
- Published: 2026-11-13
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/an-ai-search-for-memory-designs-finds-two-simple-habits-that-help

## The study

Discovering Effective Agent Memory Designs through Program Evolution (preprint, not peer reviewed). arXiv 2611.17068. https://arxiv.org/abs/2611.17068

## At a glance

- What it shows: An AI that repeatedly rewrote and tested memory software for chat assistants found designs that answered questions about past conversations more accurately than published systems, sometimes at lower running cost. Two habits kept recurring: keep the original conversation text, and handle each type of question differently.
- What it does not show: It does not show these designs working in real products, outside question answering about chats, or reliably across repeated search runs. It does not count the cost of the search itself or measure speed.
- Why it matters: Many assistants now store memories of users. The study points to two cheap design changes that builders can test today, and it shows that what you ask the search to optimise changes where money gets spent.

## What the study shows

Chat assistants increasingly keep a memory of past conversations. Most memory systems are designed by hand. They usually extract facts from chats, store them and retrieve them later. This preprint, which has not been peer reviewed, asks whether an AI can design such systems itself.

The authors built a framework called MAPE. A coding model (the paper names Claude Opus 4.8) writes a complete memory program, starting from one that stores nothing. The program is tested, and a second agent reads the execution logs to explain what went wrong. The coding model then proposes the next version. Each run lasted 20 steps. One mode chased accuracy alone. The other chased accuracy and running cost together.

They tested on two public benchmarks of question answering over long conversations, LoCoMo and LongMemEval. An AI judge marked the answers. The comparison set was seven baselines: feeding in the full history, plain retrieval of similar messages, and five published memory systems, including Mem0 and Zep.

The authors report that every selected evolved program beat the baselines on its own benchmark. On LongMemEval, the best evolved program scored 77% against 66% for the strongest baseline. On LoCoMo with an open Qwen model, the cost-aware program beat the best memory system by about 12 percentage points at about a third of its running cost.

Two patterns recurred. First, the programs kept the raw conversation text as searchable evidence, not only extracted facts. Second, they sorted questions by type and handled each differently. For a date question, code works out the date; for a counting question, code does the counting. Removing raw text from one program cut accuracy by up to about 12 points. Adding the question-routing module to five published systems raised accuracy in all five, by about 3 to 9 points.

The search goal shaped the designs. Accuracy-only search kept adding expensive steps for small gains. Cost-aware search moved work away from model calls when memory is built, towards selective work when a question arrives.

## Our reading

In our reading, the most useful result is not the evolved programs themselves. It is the two patterns, and the fact that one of them transferred. A module that helped five independently built systems is a stronger signal than a top score on one benchmark.

A practical example: an assistant is asked "When did I join that support group?" A user once said "last week". Many memory systems store the phrase "last week" and leave the model to resolve it. The evolved cost-aware program converted it to a real date before answering. When the authors swapped in a different answering model, the program that left dates unresolved fell from about 80% to 44% on date questions. The one that resolved them held at about 71%.

The evidence is narrower than the headline numbers suggest. There are two benchmarks, an AI judge with no human check, few search runs and no confidence intervals. The final programs were also chosen by their scores on the same set used to report results.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/an-ai-search-for-memory-designs-finds-two-simple-habits-that-help

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
