Testing Serving on Sites We Don't Own

Our script runs on customer pages. So the honest test of a change is whether it works on a real page, in a real browser, against real config.

That's awkward, because we can't edit a customer's HTML to point at a dev build.

So we made a bookmarklet. Click it on any page and it injects our serving script from whichever environment you pick: dev, a specific dev slot, staging, or prod. It removes any existing copy first, adds a cache-buster, and flags the injection so the script's own duplicate-guard doesn't reject it.

We only found that by doing it. The script defends against being loaded twice, which is correct in production and exactly wrong when the entire point is to replace the copy already there.

The half that mattered more

The bigger unlock was pointing a dev or staging script at production data.

Automatic tailoring works best against a production replica, because that's where the real experiment config lives. But when we created tests in prod replica on customer sites, there was no way to preview them. The extension was pinned to prod replica through staging, and the two halves couldn't be combined.

Now a checkbox threads one parameter from the bookmarklet through the script route, into the payload fetch, and onto the runtime calls, where a gate wraps the request in a prod-replica database context. You get prod-shaped experiments served onto a real page while pointed at dev.

Two guards keep that from being dangerous:

Why bother

A system that runs inside someone else's page needs a way to be exercised inside someone else's page. Unit tests don't reach that. Neither does a staging site you built to be convenient.

Most of the serving bugs we've shipped only appear on a real page: a duplicate-guard firing when it shouldn't, a stale CDN copy, a framework rebuilding a node we changed. None of those turn up in a test suite.