AI trading agents beat no simple rule; their only edge was memory
A preprint tested five language models as stock traders around earnings and bad news. None beat a plain rule, and the few gains that appeared came from what the models had seen in training.


What the study shows
The author tested AI agents built on five language models from four vendors as stock traders. The setting was a replay of past market data, 56 large US stocks in 2020, 2022 and 2026, with realistic order fills and trading costs. The paper is a preprint and has not been peer reviewed.
The first test came before 598 scheduled earnings announcements. Agents read recent headlines, prices, earlier results from peer companies, short-selling data and insider trades, then chose to buy, sell short or stay out. Most of the time they just bought. Their returns tracked a rule that buys before every announcement, which made money in 2020 and lost it in 2026.
No agent beat the simple controls once the author corrected for running many comparisons. A logistic regression, a basic statistical model fed the same numbers but no text, did at least as well as every agent. Splitting the decision into stages, or adding peer, short-selling and insider data, changed little. The agents guessed the direction of the share price reaction about as well as a coin.
The second test came after 409 bad-news days on which a stock fell 3% or more. Agents judged whether the market had overreacted and decided whether to buy the dip. The Gemini-based agent called almost every fall an overreaction and bought 96% of them. No agent's picks beat buying every drop. Their warnings of a further fall were barely better than ranking stocks by how far they had already dropped.
The only gains looked like memory. Each decision was made twice: once with the company name and dates, once with them hidden. Given only a ticker and dates, the GPT-based model called the direction of 59.2% of announcements from before its training cutoff, the date its training data ends, but only 53.3% after it. On events from July 2026 on, after every model's cutoff, no agent made reliable money in either test.
Our reading
In our reading, the most useful result is about method, not markets. The author built agents that reason like an analyst, then compared them with the dullest possible baseline. The analyst-style reasoning mostly collapsed into that baseline. An agent that buys before nearly every earnings report is running a known, cheap strategy at the price of many model calls.
The memory finding matters beyond finance. A language model has read news about past quarters and past crashes. When it is shown a company name and a date from that past, it can lean on what it remembers. The paper shows this directly: a model given no news at all, only a ticker and dates, called past earnings moves better than any agent that read the actual inputs. That advantage faded on events after its training data ended.
A concrete case: a vendor pitches an earnings agent backed by a strong backtest on well-known stocks from 2020 to 2024. On this study's evidence, that backtest cannot separate skill from recall. The same agent would need to be run with names hidden, and on events after its cutoff, before the profit means anything.
Read the rest of this story
Membership is free. You get every article in full and the Issue every Sunday.
We send a sign-in link, no password needed. Already a member? Sign inReadiness assessment
| Level | Status | Evidence |
|---|---|---|
| I Lab only | Met | Lab study of LLM agents on replayed historical data with simulated fills and costs. |
| II Benchmarked | Met | Results are scored on the public benchmark rta-bench-1, against simple rules, a logistic regression and random-stock nulls, with a clean test after all cutoffs. The results are negative, but the benchmarking is done. |
| III Open source | Met | The paper says the scoring code for both tests runs on rta-bench-1 and cites an open-source package at github.com/jackson-cmd/risk-aware-trading-agents (version 0.8.0). I could not check the repository or its licence from the paper, and whether the event-test code and saved model outputs are included is not stated. |
| IV In production | Not yet | No named organisation uses these agents in production. The paper's finding is that none earned reliable money. |
What would move it up
Level 4 needs documented live or paper trading by a named organisation, with results beating the simple bet on events after the model cutoffs. Within the paper's own frame, a larger clean test, repeated runs per model and a wider stock universe would make the evidence stronger.
The study
Jackson Wang. The Only Edge Was Memorized in Training: LLM Trading Agents Around Earnings and Bad News. arXiv, 17 November 2026. Read the paper
Notes
- Wang, J. The Only Edge Was Memorized in Training: LLM Trading Agents Around Earnings and Bad News. arXiv preprint, 17 November 2026, not peer reviewed. CC BY 4.0. Source
- Open-source package and rta-bench-1 leaderboard cited by the paper (version 0.8.0). Contents of the event tests not confirmed. Source
- The underlying benchmark, rta-bench-1, is described in an earlier paper by the same author: LLM trading agents earn beta, not alpha. Source
- Earnings test: buying before every announcement earned +18.1 basis points (hundredths of a percent) per announcement after costs (t = 0.86): +71.2 bp in 2020, +4.1 bp in 2022, -32.6 bp in 2026. The logistic regression earned +22.1 bp. Smallest Holm-adjusted p value of the multi-step agent against the five controls: 0.64. Direction accuracy of agents: 47.3% to 54.5%, with 54.2% needed for significance.
- Bad-news test: buying every drop beat the 56-stock average by +0.94% over 20 days (t = 2.02), four-factor alpha t = 0.85. Gemini 3.5 Flash's four-call agent judged 368 of 385 falls overreactions when named and bought 96% of drops. Smallest Holm-adjusted p over 75 comparisons: 1.00. Further-fall warnings: ranking by size of fall AUC 0.56, blind agents at most 0.58, named up to 0.63.
- Memory: the GPT-5.6 Luna probe, given only ticker and dates, called 59.2% of directions before its cutoff and 53.3% after. Gemini 3.5 Flash's named multi-step agent earned +32.6 bp per announcement before its cutoff and -25.5 bp after; named minus blind was +30.3 bp before and -22.0 bp after. Clean test from July 2026: always-long earned -61.2 bp per announcement; buying all drops earned -0.36% over 20 days.
Edited and signed by
Stefan PerdalEditor and publisher86 stories editedStefan read the study, set the readiness level and approved this story. An AI system found the paper and wrote the first draft.
How this was made
An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.
How we workNo corrections

