Benchmark Almanac

WorldGUI

Not in model cards

Desktop GUI agents do 611 tasks that start from varied initial states, such as the target app not being open. Tasks come with instructional videos and project files.

127 GitHub stars at 2026-11-01.

In the authors’ words

Recent progress in GUI agents has substantially improved visual grounding, yet robust planning remains challenging, particularly when the environment deviates from a canonical initial state. In real applications, users often invoke assistance mid-workflow, where software may be partially configured, steps may have been executed in different orders, or the interface may differ from its default setup. Such task-state variability is pervasive but insufficiently evaluated in existing GUI benchmarks. To address this gap, we introduce WorldGUI, a benchmark covering ten widely used desktop and web applications with tasks instantiated under diverse, systematically constructed initial states. These variations capture realistic human-computer interaction settings and enable diagnostic evaluation of an agent's ability to recover, adapt plans, and handle non-default contexts. We further present WorldGUI-Agent, a simple and model-agnostic framework that organizes planning and execution around three critique stages, improving reliability in dynamic environments. Experiments demonstrate that state-of-the-art GUI agents exhibit substantial performance degradation under non-default initial conditions, revealing limited robustness and fragile planning behaviors. Our benchmark and framework provide a foundation for developing more adaptable and reliable GUI agents. The code and data are available at https://github.com/showlab/WorldGUI.

Abstract of WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Point (arXiv:2502.08047, CC0)
Made by Show Lab , National University of Singapore Released Feb 2025 Desktop apps All tasks public
Level
Everyday Most adults, without training
Scored
Checked automatically A program checks the result against an answer key, unit tests or the expected final state.
Answers
Agent task Many steps in a terminal, browser, desktop, app, game or robot.

No scores here yet

No lab, leaderboard or Epoch AI run has reported a score for WorldGUI yet.

No lab model card reports it yet

We read the headline results of 479 launch posts from 51 labs, and none of them list this benchmark.

More in Computer and browser use

All 56 in Computer and browser use
  • BrowseComp In use Hard-to-find facts that need persistent web browsing. Short verifiable answers graded for exact correctness. Claude Fable 5 93.3%
  • ScreenSpot-Pro In use GUI grounding on high-resolution professional-software screenshots. The model must locate the target UI element. GPT-6 Astra 92.7%
  • AndroidWorld In use Agents complete 116 tasks across 20 real Android apps in an emulator. Each task checks the device's system state to judge success. GPT-6 Astra 100%
  • OSWorld 2.1 In use Agents complete 108 long-horizon workflows across 31 self-hosted websites and desktop apps, averaging 318 tool calls. Scored by binary completion and checkpoint partial credit. Claude Opus 5.5 81.8%
  • WebArena-Verified In use Agents complete 812 audited WebArena tasks on self-hosted websites. Deterministic evaluators check the agent's answer and captured network traces. Muse Spark 1.1 69%
  • MobileWorld In use Agents complete 201 tasks across 20 mobile apps, including tasks that need user clarification or MCP tools. Backend checks verify success. Kimi K3 74.4%