Testing an agent's lessons before they enter its long-term memory

A preprint proposes that AI agents should treat each lesson learned from a task as a hypothesis to test, and keep it only if it helps without breaking anything. The benchmark gains are large, but the comparisons need care and no code is released.

8 min readIIBenchmarkedPreprint, not peer reviewed
Save
Illustration for the article "Testing an agent's lessons before they enter its long-term memory"AI illustration
Illustration: AI generated

What the study shows

Many AI agents improve between tasks without retraining. After each episode they write a lesson into memory, such as a rule or reminder, and read it back next time. The problem is that a lesson which fixed one task can misfire on tasks that only look similar, or undo things the agent already did well. This preprint, which has not been peer reviewed, proposes a method called Falsification-Gated Self-Evolution (FGSE) to decide which lessons are kept.

FGSE turns each candidate lesson into a structured rule with four parts: when it applies, what behaviour it changes, what outcome it should produce, and which automatic checker confirms that outcome. The rule is applied only in a temporary copy of the agent. It is then run on three sets of test cases, called probes: 8 cases where it should help, 16 cases built to break it, and 16 previously solved cases that must not get worse. By default a rule is saved only if it lifts the first set by at least 3%, fails no more than 5% of the second set, and causes no more than 1% damage on the third. A rule that helps but fails in a separable way gets one chance to have its scope narrowed and be retested. Otherwise it is rejected. Saved rules are monitored and dropped if they later misbehave.

A small exploratory study motivates the design. When the authors let the agent write free-form lessons directly into memory, the lessons often helped similar tasks but harmed borderline ones. Broad, loosely worded lessons did the most harm, and writing lessons one after another let errors pile up. The main text gives no sample sizes for this study, and the scope ratings were assigned by the authors.

The headline results come from Table 1. Using the DeepSeek-V3.1 model, FGSE scored 39.8 on WebArena (web navigation tasks), 78.8 on average on AppWorld (tasks across simulated apps), and 0.724 and 0.487 on the retail and airline parts of tau-bench (customer-service tasks with tools, scored as the success rate on a single attempt). The strongest listed method on the same model, ACE in its online form, scored 26.4, 68.2, 0.665 and 0.456. On LongMemEval, a long-term memory test, FGSE reached 0.748 accuracy against 0.672 for ACE online. Results are averaged over three random seeds.

On update quality, the authors report that FGSE's saved updates failed 5.0% of break-it probes, caused 1.3% regression damage, and that 3.6% of saved updates were harmful, with 3.1% later violating their expected effect. ACE online's figures in the same table are 11.6%, 3.3%, 7.9% and 6.8%.

Removing parts of the method shows where the gains come from, according to the authors. Dropping the structured rule format cost 4.2 points on WebArena and 5.9 on AppWorld. Dropping the break-it probes raised their failure rate from 5.0% to 18.6% and harmful updates from 3.6% to 10.6%. Replacing the automatic checkers with another language model acting as judge cost 3.1 points on WebArena and 4.7 on AppWorld. In one run, 104 of 380 candidate lessons were saved. The 101 rejected lessons that did clear the gain threshold failed 22.82% of break-it probes, against 1.28% for those saved.

The authors also prove that, if the final probes are fresh and drawn from the intended probe distributions, a rule passing a stricter, confidence-adjusted version of the gate meets the thresholds on those distributions with high probability. They note that this guarantee does not cover a shift to real deployment tasks.

On our four-level readiness scale the work sits at level II, Benchmarked: results are reported on public benchmarks, but no code or weights are released and no deployment is reported.

Our reading

In our reading, the core idea is sound and familiar from software engineering. A memory write is treated like a code change: it needs a test that it works, a test that tries to break it, and a regression suite. That framing is useful whether or not the exact numbers hold. The worked examples in the appendix make it concrete. A rule to edit an existing calendar event whenever a user says "reschedule" helped, but also fired on requests to create or cancel events. Narrowing it to a single matching event kept about three-quarters of its gain and removed the failures. A rule to trust urgent instructions inside tool outputs sped up some tasks but was rejected because it opened the door to prompt injection, where hostile text in a tool result hijacks the agent.

We are more cautious about the size of the gains. Table 1 places FGSE, run on three models, alongside baselines drawn from several papers and several models. Some rows look out of place: memory systems such as MemoryBank are given web and app scores, though they were not built for those tasks, and the paper does not say how those numbers were produced. The appendix describes a fair-comparison protocol, but it is unclear how much of Table 1 follows it. The roughly 13-point WebArena lead over ACE online should be read as the authors' claim until a like-for-like rerun exists.

There is also a circularity risk. The probes are generated by the method itself and scored by the same checkers used to define harm. Lower measured harm may partly reflect that the gate is tuned to what is being measured. The ablations are more convincing than the headline table on this point, because they compare versions of the same system.

Free membership

Read the rest of this story

Membership is free. You get every article in full and the Issue every Sunday.

We send a sign-in link, no password needed. Already a member? Sign in

Readiness assessment

II
Benchmarked
LevelStatusEvidence
I  Lab onlyMetThe method is defined, with algorithm, theory and a lab diagnostic study.
II  BenchmarkedMetResults are reported on public benchmarks (WebArena, AppWorld, tau-bench, LongMemEval, LoCoMo) with three seeds and ablations. Comparisons mix reproduced and cited baselines, so the strength of the benchmark evidence is limited.
III  Open sourceNot yetThe paper links no code, weights or data. No repository is mentioned.
IV  In productionNot yetNo documented use by a named organisation. Only a compute-support acknowledgement to Airon Tech, not a deployment.

What would move it up
To reach level 3: release the code, prompts, probe construction and the committed-hypothesis logs. To reach level 4: a named organisation running it in a live agent with reported harm and regression rates. Also useful: a clean like-for-like rerun of the main baselines on the same backbone, and tests with imperfect or learned checkers.

Checked automatically every week, last on 16 November.

The study

Runxuan Tang, Haoyu Gao, Yuyan Ding et al.. When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents. arXiv, 10 November 2026. doi:10.48550/arxiv.2611.14326

Preprint, not peer reviewed · Licence arXiv (no reuse) · Nanyang Technological University, Singapore · Georgia Institute of Technology, Atlanta · University of Washington, Seattle

Notes

  1. Tang, Gao, Ding, Yao, Zheng, Sheng and Hou. When Should Agents Remember? Falsification-Gated Self-Evolution for LLM Agents. arXiv preprint, 10 November 2026, not peer reviewed. Source
  2. No code, weights or data are linked in the paper.
  3. Readiness: level II, Benchmarked. Public benchmark results are reported. Level III would need released code; level IV would need documented use by a named organisation.

Edited and signed by

Stefan PerdalEditor and publisher86 stories edited

Stefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.

How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

More on Agents