Computer Use Agents Passed the 72% Human Bar. Which Desktop Tasks Are Ready?

Computer use agents now self-report scores as high as 86.1% on the OSWorld benchmark, comfortably above the 72.36% that humans scored when it launched. That number is real and it is also the wrong one to plan around. On OSWorld 2.0, where tasks take a skilled person about 1.6 hours, the best agent in the paper finished 20.6% of them. The gap between those two figures is the whole automation decision.

If you only need the short answer, here it is. Computer use agents are production-ready for short, single-application tasks whose output a script can check, such as exporting a file, changing a setting or filing email. They are not ready for long multi-application workflows, GUI-driven spreadsheet and slide editing, or any step that is hard to reverse.

What the OSWorld benchmark actually measures

A computer use agent is a model that operates software the way a person does. It reads screenshots, then moves the mouse, clicks and types. The appeal is obvious: it works on software that has no API, which describes a large share of what your team uses every day.

OSWorld is the benchmark most vendors quote for this skill. It was published in April 2024 by researchers at XLANG Lab and collaborators, and it runs agents inside a real Ubuntu desktop with real applications rather than a simulated page.

369 tasks, five domains, one Linux desktop

The original set has 369 tasks on Ubuntu, 101 of them multi-application workflows. The paper groups them into five domains: OS, Office (LibreOffice Calc, Impress and Writer), Daily (Chrome, VLC and Thunderbird), Professional (VS Code and GIMP) and Workflow, meaning tasks that span several applications.

Each task starts from a set initial state and ends with an evaluation script that checks the result. That design is the benchmark's real strength. It grades the outcome on disk, not the agent's description of what it did.

The tasks are short. Epoch AI's review found that most tasks could be completed in fewer than ten steps, and only 5% need more than 50. Saturating OSWorld means handling simple, realistic tasks like adding page numbers to a document or exporting a spreadsheet to CSV.

How the 72.36% human baseline was set

The human figure comes from computer science students with basic software skills. They had not seen the tasks or, in some cases, the applications before. They completed 72.36% of tasks, with a median completion time of about 112 seconds.

That baseline is not an expert ceiling. It is what a capable newcomer achieves on unfamiliar software. Treat "beats the human baseline" as "beats a smart intern on day one", which is useful and much narrower than the headline suggests.

The detail that matters most for planning sits in the paper's own table. Human success was flat across domains: 75.00% on OS tasks, 71.79% on Office, 70.51% on Daily, 73.47% on Professional and 73.27% on Workflow. Agents behave nothing like that, as the next sections show.

OSWorld scores went from 12% to 86% in two years

At launch, the best model in the paper, GPT-4 working from the accessibility tree, scored 12.24% overall. Its Office score was 3.58% and its Workflow score was 2.97%. Those two numbers are worth remembering, because the same two areas still trail today.

Progress since then has been fast. XLANG Lab rebuilt the benchmark as OSWorld-Verified in July 2025 after fixing more than 300 reported issues, and on rerun the best system, CoACT-1, scored 60.76%. Simular said in December 2025 that its Agent S reached 72.6%, the first claim above the human line, using a method that runs several agents and picks the best result.

By October 2026, the Steel.dev leaderboard listed Qwen3.8-Max at 86.1% and several Anthropic models between 83.4% and 85.4%. On the original benchmark, the human baseline is now behind the frontier.

Self-reported, with different step budgets

Every one of the top five entries on the Steel.dev leaderboard is self-reported. None is labelled as independently verified. The original paper allowed 15 steps per task, while the current leaderboard entries run with a 100-step budget, and the Opus 4.8 row itself notes it is not directly comparable to older entries.

That does not make the scores false. It means the line measures a moving mix of model, harness and budget, so I would not use it to compare vendors or promise a success rate.

Vendor pages now quote different versions

The confusion is getting worse, not better. Anthropic's current model pages quote "OSWorld 2.1" rather than OSWorld-Verified, with qualifiers attached. The Sonnet 5.5 page lists 80.1% marked "partial". The Haiku 5.5 page lists Sonnet 5.5 at 83.9% on an "offline subset" and Haiku 5.5 itself at 72.4% on the same subset.

Neither is wrong, but both are useless for procurement unless you read the label next to the number.

Where computer use agents still fail, application by application

The headline score hides where the misses land. The most useful public breakdown comes from Simular's Agent S2 paper, which reported per-application results on OSWorld with Claude 3.7 Sonnet as the backbone and a 50-step budget. It is from April 2025, so the levels are dated. The shape is what matters.

Read the chart as a ranking of difficulty, not a forecast of your success rate.

Spreadsheets and slides: precision work through a screenshot

Spreadsheets and presentations demand exact placement. A cell, a range, a text box aligned to a guide. An agent working from screenshots has to translate pixels into coordinates, and the original paper found that mouse click inaccuracy appeared in more than 75% of the failures it analysed.

Anthropic's own documentation concedes the point. It recommends fine-grained mouse actions and modifier keys for selecting individual cells, and warns that complex spreadsheet operations might still require multiple attempts. When a vendor's documentation tells you a task type needs retries, believe it.

Epoch AI found that on spreadsheet tasks, models with access to code execution tools score considerably higher than GUI-only agents. OpenAI's computer use guide now recommends code execution for its newest model, with the computer tool kept as an alternative. If the file can be opened by a script, let a script edit it.

Multi-app workflows: where state gets lost

A multi-application task means carrying information across windows. Pull a figure from an email, put it in a spreadsheet, attach the result to a reply. Every hand-off is a chance to lose track of what has already been done.

In the 2024 paper, the best model scored 2.97% on Workflow tasks. In the 2025 Agent S2 results it scored 18.2%, the lowest of any category, even at 50 steps. On the 15-step budget, the same agent scored 5.9%. Multi-app work is the category that needs the most steps and gains the most from extra budget, which is another way of saying it is the most expensive to run.

If your automation candidate crosses three applications, the first engineering question is whether any of those hand-offs has an API or an export, not which computer use agent to buy. Integration work removes a failure point. A better agent only makes it rarer, and the cost of the integration and permissions problem it raises is covered in the analysis of agent write access to legacy systems.

OSWorld 2.0 resets the computer use score for long work

XLANG Lab released OSWorld 2.0 in June 2026 because the original tasks had become too short to separate the leaders. The new set has 108 tasks across seven professional domains, from research and engineering to healthcare and compliance workflows.

The tasks are long. They take skilled users a median of about 1.6 hours, and 69.6% take more than an hour. The original OSWorld median was under two minutes. Grading uses an average of about 27 weighted checkpoints per task rather than a single pass or fail.

The best system in the paper, Claude Opus 4.8 with maximum thinking, completed 20.6% of tasks outright at a 54.8% partial score. GPT-5.5 plateaued near 14% while using far fewer tokens. On tasks under 45 minutes, the leading models reached 20% to 24% completion. Above 163 minutes, binary completion fell to zero for every model plotted.

A Snorkel AI write-up dated September 2026 says newer models have since pushed binary completion above 45%. That claim sits outside the paper and I could not confirm it against the leaderboard, so treat it as directional. Even at 45%, more than half of long workflows still fail.

What fails over long tasks

The paper tags each task with the challenges it contains. Cross-source reasoning appears in 42.6% of tasks, visual-spatial precision in 41.7%, and implicit-state inference in 39.8%. The lowest-scoring categories are implicit-state inference, tracking several items at once, resolving conflicting information and dynamic environments.

The authors describe agents that lose track of constraints, miss information arriving mid-task, guess rather than ask the user and skip verification. That list will be familiar to anyone who has read about the failure modes that kill agent pilots. It is the same problem, now measured on a desktop.

What OpenAI, Anthropic and Google actually ship for computer use

All three large labs now sell computer use, and their documentation is more candid than their launch posts. OpenAI added computer use to its Agents API at DevDay on 29 September 2026, according to coverage by InfoQ and developer write-ups. Anthropic offers a computer use tool in the Claude API. Google launched its Gemini Computer Use model in October 2025 for browsers, and its documentation now covers desktop agents too.

Compiled from each vendor's computer use documentation as read on 19 October 2026. The OpenAI row also draws on launch coverage, because OpenAI's own recap page could not be retrieved. Pricing is excluded because it changes faster than this table.

Notice what all three agree on. Run the agent in isolation, keep it away from credentials and sensitive data, and require a human for anything hard to undo. Anthropic's documentation closes its limitations list with an instruction not to use Claude for tasks requiring perfect precision or sensitive user information without human oversight.

That is the vendors telling you, in writing, where the readiness line sits. The prompt injection exposure behind those warnings is set out in the post on prompt injection and agent security. The same risk shows up in browser agents in the review of Comet's admin controls.

A task-level readiness map for AI agent desktop automation

Here is the map I would hand to an operations lead. It combines the per-application results, the OSWorld 2.0 length curve and the vendors' own limits. The verdicts are mine. The evidence column says what each one rests on.

Two rules generate most of this table. The shorter and more single-application a task is, the more ready it is. And the cheaper it is to check the output automatically, the less the agent's reliability matters, because a failed run is caught rather than shipped.

The second rule is the one I would argue hardest for. A 70% agent with a perfect output check is safer than a 90% agent whose mistakes look like success. OpenAI's guide makes the same point in its own words, telling developers to check the saved result before reporting success.

Where this argument is weakest

A readiness map built on benchmarks inherits the benchmarks' flaws. Three of them could move the verdicts above, and one could move them in the agents' favour.

The grader is wrong more often than you think

A July 2026 audit of 150 failure-scored trajectories across five web, enterprise and desktop benchmarks found that 15.3% of FAIL verdicts were wrong. Most were evaluators rejecting valid answers, and the rest were broken tasks. Epoch AI separately estimated that about 10% of original OSWorld tasks have serious errors that make them invalid.

So some low category scores are partly the grader's fault. Spreadsheet tasks are the obvious suspect, since XLANG's own fix list cited functionally equivalent formulas being marked incorrect. Agents may be better at spreadsheets than the 25.5% figure suggests.

The audit also cuts the other way. Among the genuine failures, verification and planning problems outnumbered grounding errors. Better click accuracy alone will not fix them, which supports the case for keeping long workflows supervised.

The application breakdown is dated

The per-application numbers come from an April 2025 agent. The models at the top of today's leaderboard score more than 30 points higher overall, and none of them publishes a per-application breakdown I could find. It is possible the spread has narrowed sharply and spreadsheets are now fine.

I do not think it has narrowed to nothing, for one reason. The same categories trailed in 2024 with a different model and a different harness, and visual precision and multi-item tracking are still among the hardest tags on OSWorld 2.0 in 2026. Three data points across three years is a pattern, not proof.

There is also a measurement problem with the benchmark itself. Epoch AI found that about 15% of original tasks need only a terminal and another 30% can swap scripts for much of the GUI work. High OSWorld scores therefore say less about pure screen operation than they appear to. Your internal tools may lack that terminal escape hatch entirely.

How to run a computer use agent pilot that tells you something

A pilot that reports "the agent completed 40 tasks" has measured nothing. You need a denominator, an independent check and a baseline. The general version of that discipline is in the production readiness tests for AI proofs of concept. It applies here with one change: the check has to look at the screen's outcome, not the agent's report.

Pick 30 real tasks from one category in the readiness map. Write the check for each before the agent runs, ideally a script that inspects the resulting file, setting or record. Run each task three times, because a single success on a nondeterministic agent tells you little.

Record four numbers per task: completion, time, steps and cost per run. Then compare against the time a person on your team takes. The benchmark reports success rates, but your business case depends on cost per verified completion, which no leaderboard publishes.

Finally, run the pilot in the environment the vendors describe: an isolated machine, a domain allowlist and no live credentials. If the task cannot be done without production access, it does not belong in a pilot yet. Building the scoring harness itself is covered step by step in the guide to building an LLM eval suite.

Frequently asked questions

Where to start this week

List the ten desktop tasks your team repeats most often, then sort them against the readiness map. Anything landing in the first two rows, with an output you can check by script, is your pilot shortlist. Anything crossing three or more applications goes on a separate list headed "integration first".

Then ask your preferred vendor one question in writing: what is your success rate on our 30 tasks, at what step budget, checked by whom? A vendor who answers with a leaderboard number has not answered it.

Related reading