Self-evolving AI agents are designed to improve by turning past interactions into reusable skills. A new research paper shows why that capability creates a different security problem: a poisoned interaction can become a stored capability that is retrieved again later, even when the future user request appears benign.
The authors of EvoSkill Injection define a threat model for this failure and introduce SARGE, a multi-agent red-teaming framework that tries to make self-evolving agents generate, escalate and reinforce malicious skills. The important architectural implication is broader than the reported attack rates. Once an agent writes executable behavior into a persistent skill store, that store becomes part of the security boundary.
The attack targets learning, not only the current response
Conventional prompt injection and jailbreak testing usually asks whether an attacker can make a model produce or execute something unsafe now. EvoSkill Injection targets a longer-lived mechanism. The attacker supplies an adversarial interaction trajectory and then frames the resulting behavior as a successful experience that should be saved and reused.
The paper evaluates three self-evolving systems, AutoSkill, Voyager and ExpeL. SARGE uses an orchestrator, attack agents and a judge to test three stages: generating a malicious skill, escalating an existing skill and reinforcing a skill so that the agent treats it as a preferred behavior. A fresh session is then used to probe whether the stored capability survives and is activated again.
The authors report pass@4 attack success rates of 43.5% for generation, 54.6% for escalation and 49.9% for reinforcement in the GPT-4o-mini-based AutoSkill setup. Those numbers are research results, not established production rates, but they demonstrate the mechanism the paper is trying to isolate.
Persistence changes the control boundary
The more consequential finding is what happens after the attack. On the paper's EvoSkillSafetyBench, the reported harmful-response rate for AutoSkill rose from 6.5% in the clean condition to 19.2% after attack. Voyager rose from 5.5% to 32.5%, while ExpeL rose from 8.8% to 15.2%. The authors also report increases across several model backbones.
For engineering teams, this suggests that a skill bank should not be treated like an ordinary cache of helpful context. It is closer to executable persistent state. A skill can encode tool-use patterns, code-based actions or decision procedures, and its effect can survive the interaction that created it.
That changes the minimum governance model. A production system needs to know where a skill came from, which interaction caused it to be created, what policy version approved it, what permissions it can exercise, which agent or tenant may retrieve it and how it can be disabled or rolled back. Without that provenance, an apparently useful optimization layer can become a durable privilege-amplification path.
Output guardrails cannot repair corrupted state
The paper tests lightweight defenses based on skill-verification and conflict-resolution prompts. The authors say these measures reduce harmful responses, but they also increase refusals and do not stop malicious skills from being generated, stored or evolved.
That distinction matters. A response filter can prevent a stored capability from manifesting in one turn without removing the capability itself. The next model, tool path or retrieval condition may activate the same poisoned skill differently. In other words, output safety and state integrity are separate problems.
A stronger design therefore needs controls at the write boundary. Skill creation and updates should be policy-checked before persistence, high-risk skills may require deterministic validation or human approval, and stored skills should carry immutable provenance and versions. Retrieval also needs scope controls so that a skill learned in one task cannot silently gain authority in another.
Self-improvement creates a supply chain inside the agent
Traditional software supply-chain security asks where code and dependencies came from before they enter a system. Self-evolving agents create a smaller supply chain inside the runtime: interactions become candidate experiences, experiences become skills, and skills later influence actions.
The paper's threat model is useful because it identifies the transition from experience to reusable capability as a security-sensitive build step. Teams that already scan third-party skills or MCP servers can still miss this path if internally generated skills are automatically trusted merely because the agent created them itself.
The practical control is not to disable learning. It is to separate proposing a skill from authorizing it. An agent may generate a candidate skill, but a different policy layer should decide whether it can be stored, what scope it receives and whether later modifications require renewed validation.
The evidence is important but bounded
This is an arXiv v1 research result, and the authors state that the paper is accepted to EMNLP 2026. The evaluation covers selected self-evolving frameworks rather than every agent architecture. It relies on constructed attack trajectories, repeated model calls and an LLM-based judge, and the authors note that stronger attacker or judge models could change the results. EvoSkillBench also covers eight selected high-risk categories rather than the full space of real-world attacks.
Those limitations mean the reported percentages should not be read as estimates of how often production agents are compromised. The stronger conclusion is architectural: systems that autonomously convert experience into reusable behavior have a persistence layer that can be poisoned, and safety mechanisms need to protect that layer directly.
For teams building long-lived agents, the security review should therefore include the entire skill lifecycle: generation, validation, storage, versioning, retrieval, execution and deletion. Once agents can learn their own tools and routines, governing what they remember is no longer enough. Organizations also have to govern what they are allowed to become.