What decides how much of a job an agent can finish isn't the model. It's whether the agent can run what it just wrote and look at the result.
Our code changes our customers' web pages live, in their visitors' browsers, and there's a Chrome extension in the browser they use all day. We don't own any of it, we can't restart it, and when we break something we break it in front of our customer's customers.
So you can read our code and tell whether it's correct. You can't tell whether it works. We have plenty of tests. They run against pages we wrote, and our failures happen on pages we didn't.
Most software is cooking in your own kitchen. Ours is catering in somebody else's, and you don't find out what's in the cupboards until you get there.
Checking one change means running the whole product: backend, dashboard, the code we serve into customer pages, a build of the extension, its own database and URL. All of it isolated, in a container of its own. We call that a sandbox.
With one, it finds its own mistakes
Give an agent a sandbox and it writes the change, opens the page, sees the headline hasn't moved, fixes it and looks again. Over and over, before I see anything.
What reaches me is a second or third draft. The obvious mistakes are gone, because the agent caught them the same way I would have, by looking.
Without one, it hands the job to you
An agent that can't run its work can only reason about it. So it hands you its best guess, and every guess has to be checked by the one person who can still open a browser.
Which makes you the slowest part of your own setup. Not because you're reviewing too much, but because you're doing the agent's first pass for it.
The test worth applying
Which is why the question we ask about any agentic feature now is whether the agent can see what it just did. If it can't, it isn't finishing work. It's writing drafts and mailing them to you.
Written by
Chris Fong
CTO & Co-founder