# Some Thoughts on Building a Computer Use Benchmark

By Morgan Linton · Published 2026-11-07 · No. 3 · 429 words · https://blog.morganlinton.com/essays/some-thoughts-on-building-a-computer-use-benchmark/

This week I announced that I’m building a new eval suite, Computer Use v1 for [VulcanBench](https://vulcanbench.com). This is the first time I’ve branched out from agentic coding, which has been the focus since I started down this path.

But the more that I myself use agents more like teammates vs. robots, the more I can’t stop thinking about how important computer use is. And it’s also an area that still has so much room for improvement.

I think right now models are significantly better at writing code than computer use. And while I have a bunch of theories about why this is the case, I really just want to run experiments and look at data to understand it in more depth.

Dario, from Anthropic wrote a blog post back in October of 2024 that I found really inspiring, there’s one paragraph that has been running through my head for years now:

> In addition to just being a “smart thing you talk to”, it has all the “interfaces” available to a human working virtually, including text, audio, video, mouse and keyboard control, and internet access. It can engage in any actions, communications, or remote operations enabled by this interface, including taking actions on the internet, taking or giving directions to humans, ordering materials, directing experiments, watching videos, making videos, and so on. It does all of these tasks with, again, a skill exceeding that of the most capable humans in the world.
>
> — [Dario’s Blog](https://darioamodei.com/essay/machines-of-loving-grace)

What is so interesting about Computer Use is that it really is in many ways the tipping point for where agents truly become useful in the way a human teammate is. While we can build insanely sophisticated agentic coding workflows, there’s still relatively basic things that only humans can still do both accurately and safely.

And safety is the big piece I want to tackle, and where I think most benchmarks don’t quite go deep enough. Benchmarking is so often just focused on accuracy or speed/cost. For something like Computer Use, of course these things matter, but I would say that safety actually matters even more.

I would rather use a model that’s a bit slower, and maybe even a little less accurate than another model, if it’s safer…because I’m giving it access to my entire computer, and to be truly effective at Computer Use, it really does need access to everything.

Still noodling on the eval suite design, I put together a [draft charter on Github](https://github.com/morganlinton/VulcanBench/blob/docs/computer-use-charter/docs/COMPUTER_USE_CHARTER.md) for anyone that wants to take a look, likely changing a lot over the next few weeks, but it’s a start.