AdaptRubric gives GUI agents task-specific rules for judging success

GUI agents are becoming more capable at navigating apps, clicking controls and completing multi-step workflows, but training them still depends on a difficult question: how should a system decide whether the agent actually did what the user asked?

A new paper, Task-Adaptive Rubrics for GUI Reward Modeling, introduces AdaptRubric, a framework that generates task-specific judging criteria for GUI-agent reward models. The researchers argue that existing verifiers often rely on generic rubric structures or implicit model reasoning that does not adapt closely enough to each instruction. That can cause evaluators to carry checks from one task into another, miss concrete constraints in the current request, or enforce requirements the user never stated.

A coarse-to-fine rubric for each task

AdaptRubric builds its evaluation criteria in two stages. The coarse stage routes the instruction to a GUI task family and retrieves reusable criteria for that category. The fine stage then extracts the instance-specific details that matter for the current instruction, including concrete values, scopes and constraints.

The result is a compact rubric tailored to the task being evaluated rather than a universal checklist applied to every trajectory.

This matters because outcome reward models are not only used to score completed GUI-agent runs. They can also provide the reward signal used during reinforcement learning. If the verifier has the wrong definition of success, the policy can learn to optimize that flawed definition.

Reported gains in evaluation and training

According to the authors, AdaptRubric improves F1 by 3.6 points over the baseline average in offline reward evaluation under a matched image budget. When used for online reinforcement-learning optimization, it produces a 4.23-point gain in task success.

Those results point to a broader systems lesson: improving an agent does not always mean replacing the underlying model. The quality of the verifier, reward model and success criteria can materially change both how an agent is evaluated and what it learns during training.

For GUI agents, this is particularly important because apparently small details in a user instruction can determine whether a workflow is correct. A verifier that understands the broad task but misses a specific value, scope or constraint can still assign the wrong reward.

Why it matters for agent engineering

AdaptRubric shifts attention from “Can the model perform the task?” to a second question that is becoming equally important: “Can the system reliably recognize when the task has been performed correctly?”

That distinction becomes more consequential as agents operate across browsers, desktop software and enterprise applications where success often depends on satisfying multiple explicit constraints rather than reaching a visually plausible final screen.

The work also adds to a growing trend in agent engineering toward stronger evaluation infrastructure around the model itself: task-specific verifiers, explicit policies, trajectory analysis and reward systems that can distinguish superficially plausible behavior from instruction-following success.

The findings should still be treated as research results rather than a solved evaluation problem. The paper is a new arXiv preprint, the reported improvements come from the authors' experiments, and broader independent testing across GUI environments and agent architectures will be needed.

For now, the important signal is clear: better reward criteria can improve both the measurement and training of GUI agents. As agent systems become more autonomous, defining success precisely may be as important as making the model more capable.

Source
- Task-Adaptive Rubrics for GUI Reward Modeling