UiPath has added an LLM-as-Judge guardrail to UiPath Agents in preview, turning a natural-language policy into a model-backed check that can inspect agent prompts, responses and LLM calls. The release is notable because the guardrail is not a fixed rule engine: every check makes its own LLM call, and UiPath says that call consumes Agent Units or Platform Units in addition to the agent's normal usage.
That changes the engineering shape of a guardrail. A team is no longer configuring only a policy sentence and an action. It is also choosing a judge model, a decision threshold, examples, where the check runs, and whether the same check is active during evaluation. Those settings become part of the effective control boundary.
The guardrail is a model call inside the control path
UiPath's September release notes describe the feature as a built-in guardrail that evaluates agent inputs and outputs against custom natural-language instructions. The Python SDK exposes the same mechanism as LLMAsJudgeValidator, with support for checks before execution, after execution or at both stages.
The validator accepts a judge model, a text instruction of up to 4,000 characters, a threshold from 0 to 6, and up to two positive and two negative examples. The SDK documentation says lower thresholds are stricter, with 2 as the default. It also notes that the selected model must be allowed by the organization's governance policy through LLM Gateway.
This means the enforcement path depends on model inference rather than only deterministic matching. The policy text remains important, but the result is also a function of which model interprets it and how the threshold and examples are configured.
Governance now depends on the judge model as well as the policy text
For audit purposes, storing the natural-language rule alone is not enough to reconstruct how a decision was made. The effective control configuration includes at least the judge model, its version or deployment identity, the threshold, examples, execution stage and resulting action.
That is a practical governance consequence, not just implementation detail. Two deployments can use the same written rule and still produce different enforcement behavior if they use different judge models or thresholds. A change to the judge therefore deserves the same configuration discipline as a change to the policy itself: versioning, approval, regression testing and traceability.
UiPath's documentation also says platform support is still rolling out and may not yet be available in every tenant. Preview status matters because teams should not treat current behavior or availability as a stable production contract.
The cost model makes guardrail placement an engineering decision
The release notes explicitly state that every LLM-as-Judge check triggers a real LLM call and consumes units on top of the agent's existing consumption. UiPath's licensing documentation separately confirms that coded-agent LLM calls are metered in Agent Units or Platform Units depending on the licensing plan.
That makes placement significant. A check that runs only after a high-risk operation has a different cost profile from a check that runs before and after every LLM interaction. Applying several judge rules at several stages can multiply inference calls even if the underlying agent workflow is unchanged.
The useful design question is therefore not simply whether an LLM judge can express a policy more flexibly than a deterministic rule. Teams also need to decide which decisions justify probabilistic review, where a deterministic control is sufficient, and where the added inference cost and latency are acceptable.
Evaluation can be distorted if the judge is part of the system under test
The SDK exposes an enabled_for_evals setting and documents it as enabled by default. That is convenient for testing the governed system, but it creates an evaluation distinction that teams should make explicit.
If an evaluation measures an agent while the judge is actively blocking or logging behavior, the result measures the combined system: base agent plus guardrail. That can be exactly what a production-readiness test should measure. It is different, however, from measuring the base agent's own behavior.
For reproducible evaluation, Aipolix's analysis is to keep both views where they matter: a baseline that characterizes the agent without the judge, and a governed-system evaluation that includes the exact judge configuration used in deployment. Otherwise an improvement in the reported safety or compliance rate can be mistaken for a change in the underlying agent when it was actually produced by the enforcement layer.
Preview status limits how strongly the feature should be interpreted
UiPath has shipped a concrete mechanism, but the company labels it preview and says availability is still being rolled out. The evidence supports describing the control surface, configuration and metering behavior; it does not establish how reliably a judge model will enforce arbitrary policies across domains or adversarial cases.
For teams considering the feature, the immediate operational takeaway is narrower: treat the judge as a versioned, metered component of the runtime control path. Record its model and configuration alongside the policy text, test failure modes separately from the base agent, and budget guardrail calls as part of the agent's inference envelope.