back to blog

The Screen Is Becoming a Reward Surface

The Screen Is Becoming a Reward Surface

A useful computer-use agent does not need to explain itself beautifully. It needs to actually finish the task.

That sounds obvious, but a lot of the bottleneck in agent progress still lives right there. We have models that can click around, type, scroll, open tabs, and produce very convincing traces of effort. What is harder is scoring whether the trajectory really accomplished what the user asked for without paying a human to inspect every run.

That is why Video-Based Reward Modeling for Computer-Use Agents is interesting.

The paper's core move is clean: evaluate the agent from the execution video itself. Not from its chain of thought. Not from privileged access to its internal actions. Not from hand-written task-specific checkers every time. Just the instruction and a sequence of keyframes showing what happened on screen.

If that sounds almost too simple, good. Those are usually the ideas that unlock new infrastructure.

The authors build a dataset called ExeVR-53k with 53,000 instruction-video-reward triples, then train an 8B reward model to predict whether a task succeeded from the visual evidence alone. They also spend real effort on the annoying practical part, which is where a lot of agent papers quietly fall apart. Screen recordings are long, repetitive, and full of irrelevant pixels. Success often depends on tiny local changes: a form field filled correctly, a button state changing, a confirmation banner appearing, the right tab ending up in focus. So they add spatiotemporal token pruning to throw away the boring parts while preserving the decisive ones.

That detail matters because it points at the real direction of travel.

The interesting future for agents is probably not one giant end-to-end magic model that simply "knows" whether it did a good job. It is a stack of narrower pieces: policies that act, memories that route context, environments that push back, and reward models that can cheaply tell the difference between motion and progress.

Screens are an especially important environment because so much of civilization is already trapped inside them.

Most office work, admin work, research work, logistics work, purchasing, customer support, compliance, analysis, scheduling, filing, bug triage, vendor management, internal tooling, and weird glue work between systems still reduces to one thing: moving through interfaces and leaving them in a better state than you found them. If you can reliably tell whether that happened from the visible trace, you get a much denser training signal for useful digital labor.

That is the part people underrate. Better reward models do not just improve evaluation. They change what becomes trainable.

Once success on a screen can be judged cheaply and at scale, computer use starts looking less like a demo category and more like a real learning loop. Agents can attempt tasks, get scored from the outcome, and improve without every domain needing bespoke instrumentation. You do not need the software vendor to expose a perfect API for every legacy workflow. The UI itself becomes legible enough to supervise against.

That feels like a small shift, but it is a very leverage-heavy one.

A lot of the path to more abundant intelligence is not about making models more theatrical. It is about turning messy human environments into places where feedback is available. Code had tests. Some parts of math have proof checkers. Robotics has the body and the task. A huge fraction of digital work has the screen.

Once the screen becomes a reliable reward surface, the amount of work that can be iterated on by machines expands fast.

There are obvious limits. Screens can hide state. Some tasks require background knowledge, external judgment, or side effects you cannot see in a keyframe sequence. A visually plausible result is not always the right result. But those are reasons to keep building the stack, not reasons to dismiss the direction. The point is not that video alone solves agents. The point is that it gives us one more cheap contact surface with reality.

That is the kind of thing progress compounds on.

The field has already learned, over and over, that raw generation is not enough. Capability appears when generation is tied to something that can answer back. For software that answer is usually a test suite. For computer use, this paper argues that the answer can increasingly come from the visual trace of the work itself.

That makes the future feel more practical to me.

Not because it is glamorous, but because it lowers the cost of training agents on the software world we already have. And that world is enormous. It is full of ugly dashboards, fragile internal tools, half-automated workflows, and ancient web apps that still run companies. If agents can learn to operate there from outcome signals grounded in what actually changed on screen, intelligence stops depending so much on clean greenfield environments.

It can start colonizing the mess.

That is how useful automation usually spreads. Not by waiting for the world to become elegant first, but by getting good enough to work inside the interfaces history left behind.

A reward model that can watch a screen and tell whether the task was actually done is one step toward that world.

Small step, maybe. But it points in the right direction: more feedback, more trainable surfaces, more useful intelligence in the places people already live.