# Claude Code agents built post-quantum FPGA hardware, but their tests missed faults

A preprint follows AI coding agents as they port a quantum-safe encryption chip design to new hardware. The hardware works, but the agents' own tests and reviews proved weaker than they looked.

- Category: Code
- Readiness level: II (Benchmarked) on Science Underway's four-step scale, as proposed
- Published: 2026-11-20
- Editor: Stefan Perdal
- URL: https://scienceunderway.com/articles/claude-code-agents-built-post-quantum-fpga-hardware-but-their-tests-missed-fault

## The study

From FIPS to FPGA with LLM Agents: Measuring What Tests and Reviews Miss in Post-Quantum Cryptographic Hardware (preprint, not peer reviewed). arXiv 2611.25339. https://arxiv.org/abs/2611.25339

## At a glance

- What it shows: AI agents, guided by one engineer, moved a working quantum-safe encryption design onto a new circuit board, and it gave correct answers on every test it could run. But when the authors broke the design on purpose, the agents' own tests caught only about three quarters of the faults.
- What it does not show: It does not show that agents are faster or cheaper than human engineers, or that the hardware is secure against physical attacks. It covers one design, one engineer and one family of AI models.
- Why it matters: Companies must replace today's encryption with quantum-safe versions in the coming decade, and AI agents may do much of that work. This study shows that tests and reviews written by the same agents need independent checks before anyone trusts them.

## What the study shows

The authors, from Lucid Motors and EYL Inc., used AI agents in the Claude Code harness to port a hardware accelerator. The accelerator runs ML-KEM and ML-DSA, the new US standards for quantum-safe key exchange and digital signatures. It went from an older FPGA card (a reprogrammable chip) to a newer Zynq UltraScale+ card running three times faster, at 300 MHz. This is a preprint and has not been peer reviewed.

The agents worked in a seven-stage workflow. Every claim about the design became an automatic check, called a gate. A gate only counted if it was shown to fail on deliberately broken code. Separate reviewer agents then audited each piece before it was saved.

The port worked. All six operations gave byte-exact results on the board. It passed all 85 cases of NIST's official conformance tests that the device's interface could accept, out of 285 in total. It also passed 484 extra signature checks.

The authors then tested the tests. They used a standard tool to make hundreds of copies of three modules, each with one random bit of the circuit changed. These broken copies are called mutants. The existing tests caught 76% of them. For the key-exchange decryption module, the figure was only about half.

An agent then wrote five new tests from the specification, guided by the mutants that slipped through. On a fresh batch of mutants, these raised the decryption module's catch rate from 48% to 74%.

The authors also replayed AI code review on four known flaws. These were flaws that standard known-answer tests, which only check correct outputs for valid inputs, cannot see. When reviewers were pointed at the flawed part, they found every flaw in all 54 sessions. When asked to review the whole design, they often missed two flaws in the signature checker, finding one of them in only 1 of 6 sessions.

Finally, the authors planted small one-line flaws in the key-protection logic. Reviewers pointed at that part found them in only 3 of 20 sessions. Formal checks, which mathematically prove a property holds, flagged every one.

## Our reading

In our reading, the headline is not that agents can build cryptographic hardware. It is that the evidence agents produce about their own work is softer than it appears. Every test passed, every review approved, and yet a quarter of random faults went unnoticed. A product team reading a green dashboard would have seen nothing wrong.

The review results point to a practical rule. AI reviewers did well when told exactly where to look, and poorly when handed a whole design. That matters because, in real projects, nobody knows in advance where the flaw is. The study suggests keeping review narrow, one module at a time, and before every commit.

For the most sensitive logic, the part that keeps secret keys inside the device, neither tests nor AI review were enough. Only formal checks caught the planted flaws. On our Readiness Scale we rate this work at level II, Benchmarked: it reports results against NIST's public conformance vectors, but no code or designs are released.

The rest of the article (what it means, limits, what to watch) is free for members: https://scienceunderway.com/articles/claude-code-agents-built-post-quantum-fpga-hardware-but-their-tests-missed-fault

## How this was made

An AI model read the study and wrote this article in the structure Science Underway uses. An editor reviewed it and approved it for publication. Check the original study before you rely on the details.

---
Source: Science Underway, https://scienceunderway.com/. Please link to the article when you cite it.
