VeriCUA explores digital environments to check whether agent tasks are complete

What the study found

In an arXiv preprint, Qiushi Sun and colleagues present VeriCUA, a framework for verifying tasks performed by computer-using agents. The authors describe a shared procedure intended for online and offline settings across platforms and observation types.

The verifier first examines the available context and constructs a rubric of task requirements. For requirements that remain unresolved, it selects graphical user interface or code checks, then combines their results using explicit evidence rules.

The authors report that VeriCUA agreed more often with reference labels than the baselines they compared across the tested settings. They report the largest offline gains on hard cases and stronger rejection of false completion claims online. Their further analyses connect the gains to complementary evidence and exploration at test time.

Why it matters

Reliable completion checks can support the curation of training data, evaluation and reinforcement learning for agents that use digital environments. The proposed approach addresses a limitation the authors identify in existing automated checks: a vision-language model can judge only the observations supplied to it, while a generated program depends on what application state it can access. Selecting additional checks could help a verifier assess requirements that one view alone leaves unresolved.

Limits and unknowns

These findings are reported in a preprint and are based here on its abstract. The abstract does not specify the datasets, sample sizes, numerical results, baseline details or the precise tests used. It also does not establish how well the framework generalizes beyond the evaluated settings, or what computational and engineering costs its exploration entails.