Natural-language agents can produce a DeFi workflow that looks complete, contains the expected operations and even declares a slippage limit, yet still authorize a trade that loses too much value to its own market impact. A new arXiv preprint, DeFiFlowBench, turns that failure mode into an executable benchmark rather than judging generated workflows only by syntax or graph structure.
The authors evaluate 207 team-authored prompts spanning swaps, limit orders, cross-chain transfers and compositional tasks. Their central finding is practical: structural correctness and declared safeguards do not establish safe execution. A workflow can include a minimum-output rule derived from a quote while still allowing the order itself to move the price beyond an acceptable limit.
Structural correctness is only the first rung
DeFiFlowBench uses a ladder of checks rather than one pass-or-fail score. The first level asks whether the generated graph contains the required node types and edges. The second asks whether required configuration values are concrete. The third checks whether required safety predicates are declared and parameterized.
The paper then runs a separate execution check on a simplified local EVM environment. That distinction matters because a workflow can pass static checks while still producing an economically unacceptable result. The local harness uses a constant-product automated market maker and labels an executed swap unsafe when its own price impact exceeds a fixed 5% cap.
The authors report that direct, constrained and few-shot language-model prompting produced 14 to 19 unsafe held-out executions per configuration under that fixed cap. Those figures are results from the paper's controlled environment, not evidence that deployed DeFi agents fail at the same rate.
Slippage and price impact protect against different failures
The paper's most useful technical point is that slippage tolerance and price-impact control are not substitutes. A minimum output calculated from an already impact-adjusted quote can protect against market movement between quote and execution. It does not necessarily prevent a large order from causing an excessive price movement by itself.
This distinction explains why a generated workflow may look responsibly configured while still authorizing a bad trade. A generator that has learned to add a slippage field can satisfy a familiar safety pattern without constraining the economic effect that matters in that transaction.
DeFiFlowBench therefore evaluates an explicit price-impact gate separately. It also tests cases where a generated workflow already contains permissive thresholds, because simply injecting defaults does not help when an unsafe value is already present.
Koan-Safe moves protection outside the generator
The authors propose Koan-Safe, which separates intent parsing, candidate generation and a generator-independent enforcement layer. The enforcement step can repair missing structural safety components and add defaults such as a slippage bound, a price-impact gate, bridge confirmations and an order expiry when the candidate omitted them.
On 75 held-out workflow prompts, the paper reports a 0.67 score on its static safety proxy for the hybrid Koan-Safe variant, compared with 0.33 for the best baseline. The saved benchmark outputs recorded no unsafe executions under the corrected local checker. When the authors disabled enforcement while holding the candidate constant, they report 14 to 17 unsafe executions.
These are promising experimental results, but the paper is careful about their scope. The execution harness does not run the full generated workflow graph against a live protocol. Tokens, reserves and routing are simplified, and a high static score or zero observed unsafe trades is explicitly not presented as a general safety guarantee.
The policy cap is the more important engineering lesson
The strongest result is not that one repair layer achieved a higher benchmark score. It is what happened when existing thresholds were already too permissive. Koan-Safe's default injection only fills missing values; it does not automatically clamp every unsafe threshold that a generator or user has supplied.
The authors therefore evaluate a separate policy cap on a 36-case diagnostic grid. That additional control addresses a class of failures that structural repair alone cannot solve.
Aipolix's analysis is that this is the part agent builders should generalize beyond DeFi. A generated workflow should not be allowed to define the entire safety envelope that will judge the same workflow. The generator can propose parameters, but execution should remain subject to an external policy that establishes non-negotiable limits independently of the generated plan.
In financial automation, that can mean a hard price-impact ceiling. In infrastructure automation it could mean a maximum blast radius, protected resource classes or change windows. In data systems it could mean limits on disclosure scope. The shared design principle is that semantic or structural validity is not enough when the action has a measurable external cost.
The benchmark is useful, but the evidence is still preliminary
DeFiFlowBench is an arXiv v1 preprint, not a peer-reviewed or independently replicated result. Its benchmark is team-authored, and the execution environment intentionally simplifies real DeFi behavior. The local checker does not establish that a workflow fulfills the user's intent or that it would remain safe against deployed protocol behavior, adversarial liquidity conditions, MEV or other production effects.
The paper links the existing Koan repository as its code location. The repository describes an executable no-code DeFi platform, but its current top-level presentation does not make the benchmark and Koan-Safe artifact as easy to inspect independently as the paper's detailed methodology suggests. That limits reproducibility today even though the paper retains saved outputs and describes its evaluation protocol.
For practitioners, the immediate value is therefore architectural rather than a claim that Koan-Safe is production-ready. Agent evaluation should include execution consequences, and the final execution boundary should enforce policy limits that the generated workflow cannot silently weaken.