K-Bench tests a privacy assumption that becomes fragile once a language model is wrapped in an agent: if an unlearning benchmark no longer sees a secret in the final answer, has the deployed system really forgotten it?
Researchers from the University of Technology Sydney and CSIRO built K-Bench, an open benchmark for machine unlearning inside a ReAct-style agent. Instead of checking only the final answer, it observes six surfaces exposed during an agent run, including reasoning traces, tool calls, tool observations, retrieval output, the answer and an elicited summary.
The distinction matters because a defense can make one surface look clean while the same information remains visible somewhere else in the execution path. In K-Bench, when a secret is stored in the prompt or retrieval layer, conventional model-level benchmarks such as TOFU and MUSE report no leakage because they probe the model's parametric memory. The deployed agent still exposes the target information on 22% to 86% of queries, depending on the substrate and configuration.
A clean answer can hide a dirty execution trace
The paper's motivating example is a structured retrieval task. A filtering method alters the final answer so the true date of birth no longer appears there. An answer-only test therefore treats the target as suppressed.
But the agent has already asked a record lookup tool for that person's date of birth, and the tool observation contains the correct value. K-Bench marks the run as leaking because it treats exposure on any observed channel as exposure of the secret.
That is a more operational definition of privacy for tool-using systems. The output seen by the user is only one boundary. Tool arguments, tool results, retrieval context and intermediate state can all carry the information that a deletion or unlearning control was supposed to remove.
The benchmark separates four memory substrates: model parameters, prompt context, free-text retrieval and structured retrieval. Each experiment puts the secret into one substrate so that the source of a leak can be isolated rather than inferred.
Twenty published methods still leave a hard problem
K-Bench also evaluates published unlearning methods on the parametric substrate, where weight-based techniques are actually designed to operate.
The paper reports that none of twenty evaluated published methods demonstrably removes the target secret under its multi-channel observer. Some methods reduce visible leakage but damage the agent's ability to function. Others resist the tested extraction while the authors cannot verify that the underlying knowledge was removed.
Refusal is not the same as forgetting, and agent collapse is not a successful privacy control.
The authors therefore use a collapse-aware score that considers forget suppression, retained behavior and agent stability together. They also report that the highest-ranked method changes across base models, which makes a single-model leaderboard a weak basis for a general claim about which unlearning method is best.
The study covers Llama-3.1-8B, Mistral-7B-Instruct-v0.3 and Qwen3.5-9B in the main cross-model work. The benchmark release includes evaluation code, synthetic PII splits, a pre-registered statistical plan, baseline traces and retrieval assets.
The strongest result is also narrower than the headline
The paper is careful about what it has established. Its main corpus uses synthetic personally identifiable information generated with Faker. A real-format validation using the LUME benchmark is limited to dates of birth because that is the attribute supported cleanly by the study's record tool. The parametric experiments inject the target information through LoRA fine-tuning rather than testing secrets memorized naturally during pretraining.
The authors also test pure substrates one at a time. A real deployment can hold the same fact simultaneously in model weights, prompts, retrieval stores, caches, logs and application databases. K-Bench does not claim to have measured every such combination.
There is another important portability limit. Several published unlearning implementations had to be adapted to run in the benchmark, and nine ports differ from their original releases in ways that could affect a rerun. The authors document those changes rather than hiding them, but they make the result a controlled comparative study, not a final ranking of the entire unlearning field.
Those caveats reduce the scope of the claim. They do not erase the central systems result: checking only the model's answer is not enough to certify forgetting in an agent whose execution exposes additional channels.
What this changes for agent privacy architecture
Aipolix's analysis is that K-Bench shifts the unit of verification from the model endpoint to the whole execution boundary.
For a production agent, a deletion request cannot be treated as complete merely because the model stops repeating a fact. Teams need to know where that fact can still be reached: model parameters, prompts, vector stores, structured databases, tool responses, summaries, traces and other persisted execution state.
That leads to a more demanding control model. Privacy enforcement should combine source-level deletion or revocation with runtime minimization, explicit provenance and channel-aware tests. Where data cannot be physically removed from every layer, the system needs enforceable access controls and evidence that blocked information cannot be reconstructed through another path.
K-Bench also exposes a governance problem for benchmarks themselves. A compliance dashboard that measures only the final answer can produce a reassuring score while the agent's tools still process or reveal the same data internally. The measurement surface has to match the deployment surface.
This is especially relevant for systems that use RAG. Removing a fact from model behavior does nothing to a retrieval store that still contains it. Conversely, deleting a vector record does not prove that copies are absent from prompts, summaries or model parameters. Unlearning and data lifecycle controls are related, but they are not interchangeable.
An open benchmark, not a compliance certificate
The authors explicitly connect K-Bench to data-erasure obligations such as GDPR Article 17, but the benchmark is not itself proof of regulatory compliance. It is an auditing instrument for a narrower technical question: whether target information remains recoverable through the channels the harness observes after an intervention.
The benchmark is useful because its assets and statistical protocol are public, and because it forces methods to preserve utility instead of winning by breaking the agent. It is still a preprint, its real-format validation is limited, and its observer does not reconstruct secrets by combining partial fragments across multiple channels.
The practical lesson is stronger than "unlearning does not work." The better conclusion is that unlearning cannot be evaluated in isolation from the agent architecture that surrounds the model.
If an agent can retrieve, transform or pass a supposedly forgotten fact through another component, a clean final answer is only a clean interface. It is not evidence that the system has forgotten.