Experiments Workflow

Starting an Experiment

The default experiment is a 50/50 A/B test between your original page and a Tailored page variant. Hit Start Test, confirm the details, and you're live. That's it.

The Default (Recommended)

50% of traffic sees the original page (control), 50% sees your Tailored variant (treatment). This is the fastest way to learn and works for the vast majority of experiments.

Advanced Experiment Options

For more complex tests, Tailor supports additional configurations. Any of these advanced changes will move the experiment from the standard Tailored page view into the Advanced Experiment Tests section.

A/B/C or A/B/C/D Tests

Add additional Tailor page variants to test against. For example, test three different headlines at once. This splits traffic across more variants, so you'll need more volume to reach significance.

Custom Control

By default, the control is the original base page. You can change the control to be another Tailor page variant instead. This is useful when you've already promoted a winner and want to test a new idea against it rather than the original.

When to Use Advanced Options

Only add extra variants when you have enough conversion volume that splitting traffic three or four ways won't slow results to a crawl. Variants should be meaningfully different, not minor tweaks.

If you're struggling to get steady conversions with two variants, stick to a simple A/B test. Sequential iteration (promote the winner, then test the next idea) is almost always better than testing many variants at once.

Reading Results

Use this five-point framework to evaluate your experiment:

1Primary goal: did the variant improve meaningfully vs. control on your conversion goal?
2Volume: do you have enough conversion volume to trust the direction?
3Consistency: does the lift hold across days and major segments?
4Diagnostics: if clicks are up but downstream conversions aren't, the variant may be driving engagement without moving visitors to convert. This is useful signal for your next iteration.
5Confounds: did campaigns, deploys, or tracking change mid-test?

Statistical Significance & Confidence Labels

Every experiment gets a confidence label based on the statistical significance of the results so far. You can hover over any label to see the exact confidence score (e.g. 78% or 96%).

Too Early

Not enough impressions or conversions to draw any conclusion yet. Need to wait a little longer.

Low Confidence

Some data is in, but not enough to trust the direction. Keep waiting.

Early Signal

Getting closer to statistical significance. The direction is starting to become clear, but not yet conclusive.

High Confidence

Over 90% statistical significance. You can trust the result and act on it.

How Long Do I Need to Wait?

The time to reach significance depends on two factors:

Traffic and conversion volume

More visitors landing on the page and converting means faster results.

Size of the difference between variants

If the change has a large impact on results, you won't need much traffic to detect it. If the change is subtle, you'll need a lot more traffic to confidently measure the difference.

How to Speed Up Results

Increase conversion volume by driving more qualified traffic
Reduce variants (A/B instead of A/B/C) to concentrate traffic
Make bolder changes that are more likely to produce a measurable difference
Temporarily use a higher-frequency proxy goal (e.g. form starts instead of form submits)

If results aren't accumulating after a few days, check that your conversion goal is firing correctly on both control and treatment. This is the most common setup issue and takes minutes to verify.

Minimum Traffic Needed

There's no universal minimum, but the rule of thumb is: you need enough conversions that you're not reading noise. If your page converts at 2% and you get 100 visitors/week, that's about 2 conversions per variant per week. It will be very hard to detect lift at that volume.

For low-volume pages: stick to A/B (not multi-variant), make bigger and more meaningful changes, and consider a higher-frequency proxy goal to validate direction before committing to a longer test.

Promoting a Winner

When you're confident in the results, here's the workflow:

Confirm lift on primary goal

Make sure the improvement is on your real conversion goal, not just clicks or engagement metrics.

Ramp to 100%

Click "Ramp to 100%" to send all traffic to the winning variant.

Start the next test

Use the winner as the new control and test one new hypothesis. Don't pile multiple changes into one test.

Monitor for regression

Monitor metrics for a few days after promotion to confirm the lift holds. Early results can be influenced by traffic spikes or seasonal patterns.

Stopping a Test Safely

To stop an experiment without losing data:

1Deramp the variant (set allocation to 0%). This stops exposure without deleting anything.
2Confirm the control experience is serving normally.
3Record the final readout: dates, traffic, conversions, and lift.
4Decide: promote the winner, iterate on the variant, or scrap it.

Deramp vs. Delete

Always deramp first. Deramping keeps the variant for future reference or re-testing. Only delete if you're certain you won't need it again. Available experiment actions are: ramp, ramp to 100%, deramp, and delete. There is no pause action.

Scheduling Experiments

Experiments launch instantly when you click Start Test. Most teams launch and monitor in real time. If you need to coordinate timing, QA your experiment ahead of time and launch when ready.