Your A/B Tests Are Running Longer Than They Have To. CUPED Fixes That.

If you run more than a dozen A/B tests a year, test duration is probably the biggest drag on your experimentation velocity. Most teams try to fix it by bumping traffic allocation, cutting the number of variants, or calling the test early when it "looks significant." All three are the wrong answer. There's a better one that most experimentation teams outside big tech still aren't using: CUPED.

Most of Your Test Variance Isn't From Your Test

Here's the thing people miss. When you measure a metric like revenue per visitor or checkout conversion rate in an A/B test, a lot of the variation you see across users isn't from your treatment. It's pre-existing user heterogeneity. Power users who've been converting for months. Seasonal shoppers. First-timers who bounce regardless of what you show them. That variation was there before your test started. It has nothing to do with what you're testing. But it inflates your standard error and forces you to run longer to reach significance.

CUPED (Controlled Experiments Using Pre-Experiment Data) was originally published by Alex Deng, Ya Xu, Ron Kohavi, and Toby Walker at Microsoft in 2013. The core idea: if you have data on user behavior before the experiment started, you can use it to subtract out the portion of variance that was predictable all along. You're left with a cleaner signal that's easier to detect with fewer users and fewer days.

How It Actually Works

You take a pre-experiment covariate, usually the same metric you're measuring but from a lookback window before the test started. You run a simple linear regression to estimate how much of each user's post-experiment outcome is explained by their pre-experiment behavior. Then you subtract out that predictable portion.

The adjusted outcome looks like this:

Y_adjusted = Y - θ * (X - E[X])

where X is the pre-experiment covariate, E[X] is its mean, and θ is the regression coefficient from your linear model. Y_adjusted has lower variance than Y by a factor of (1 - ρ²), where ρ is the correlation between the covariate and the outcome.

At Bing, the team found they could reduce variance by roughly 50%. Tests that previously took 8 weeks could be completed in 5 to 6 weeks. That's not a minor optimization. That's one or two extra tests per quarter without adding traffic or relaxing your statistical thresholds.

The Correlation Is Everything

CUPED only delivers meaningful variance reduction if your covariate correlates well with your metric. For most engagement metrics, a 14 to 30-day pre-experiment lookback on the same metric will give you strong correlation. Amplitude Experiment defaults to 14 days. Some teams standardize on 42 days for metrics with more weekly volatility.

The practical rule: if ρ² is around 0.3 or higher, you'll see worthwhile reduction. Below 0.2, the math is working but the win is marginal. Check it before you trust the shorter runtime.

Where this covariate quality breaks down:

• New user flows. First-time visitors have no pre-experiment history. You either find a proxy covariate (device type, acquisition channel) or accept that CUPED won't help much here.
• New features. If the feature didn't exist before the experiment, there's no direct covariate. You need a surrogate metric, and that weakens the reduction.
• Low-frequency events. If only 2% to 3% of users trigger the outcome (like completing a high-value action in a thin segment), the correlation between pre- and post-experiment behavior often isn't strong enough to matter.

I haven't run CUPED on every edge case, so I'll be direct: for most tests targeting returning users on established flows, it works well. For new user acquisition or product launch tests, manage your expectations.

Where It Falls Apart Completely

CUPED is not a fix for pre-experiment imbalance. If your randomization was flawed, if control and treatment had meaningfully different baselines before the test started due to an SRM or a bad holdout setup, CUPED doesn't solve that. It reduces noise in the outcome metric. Systematic bias in group assignment is a different problem that needs fixing upstream.

Also: CUPED doesn't justify shortening your minimum detectable effect threshold or running underpowered tests. It gives you faster results for the same power, not the same results from lower power.

Platform Support in 2026

Most major experimentation platforms now have CUPED built in. Statsig, Amplitude Experiment, Optimizely, Eppo, and GrowthBook all support it natively, though some have it off by default. If you're on one of these, check your experiment settings before assuming it's active.

Adobe Target doesn't expose a native CUPED option. If Target is your primary testing tool, you'd handle variance adjustment in your analysis layer by pulling raw event data into BigQuery or a notebook and doing the regression there. It adds a step, but it's not complicated. A Python script using statsmodels or scipy takes about 20 lines.

The Quick Takeaway

If your experimentation platform supports CUPED and you haven't turned it on, you're almost certainly running tests longer than you need to. The fix is simple: enable it, verify the covariate correlation is meaningful (ρ² above 0.2 at minimum), and use a 14 to 30-day lookback for stable metrics.

The biggest gains show up on returning-user flows, revenue per user, and session engagement metrics where historical behavior is a reliable predictor of future behavior. Those also happen to be the metrics that take longest to reach significance in the first place. That's where the compounding effect is.

You can read the original CUPED paper from Microsoft Research if you want the full statistical derivation. It's more readable than most academic stats papers.