What is time-based regression?

Time-based regression is a method for analysing geo experiments that regresses the treated markets' aggregate metric on the control markets' over the pre-period, then projects that relationship through the test window. The gap between actual and projected values, accumulated over the window, is the estimated lift.

Also called
TBR, TBR model, time-based regression for geo experiments
Allon Korem

Written by Allon Korem

Chief Executive Officer

Last updated

In plain English

Time-based regression, usually shortened to TBR, was set out by Google researchers in 2017 as a way to analyse geo experiments. Rather than modelling each market separately, it aggregates the treatment markets into one series and the control markets into another. Over the pre-period it fits a single linear regression of the treated series on the control series. Through the test period it applies that fitted relationship to what the control markets actually did, which gives the counterfactual forecast — the path the treated markets would have taken had nothing changed — and the accumulated difference between actual and forecast is the lift.

The appeal is that the controls carry everything the treated markets share with them. A national promotion, a holiday, a heat wave or a competitor's launch moves both series, so it moves the forecast along with the actual and drops out of the difference. What remains is what happened only in the treated markets during the window, which in a well-run test is the treatment. Compared with difference-in-differences, which assumes the gap between the groups stays constant, TBR lets the treated series scale with the controls through the slope. Compared with synthetic control, it fits one coefficient on an aggregate rather than a weight per donor market, which makes it hard to overfit and easy to inspect.

Uncertainty is the part implementations differ on. The original paper used a Bayesian regression and reported a predictive interval on the cumulative effect, which widens through the test window because each day's forecast error accumulates. Meridian GeoX keeps the regression, constrains the slope to be non-negative, makes no distributional assumption about the errors, and takes its intervals from placebo tests on periods where nothing happened. Either way the interval should grow with the length of the test; one that stays flat is treating the forecast as if it were known.

It breaks in three recognisable ways. Contaminated controls: if control markets receive spillover from the treatment — overlapping broadcast coverage, or national campaigns that reach some regions harder than others — the counterfactual absorbs part of the effect and the lift is understated. A relationship that drifts: the method assumes the pre-period slope holds through the test, so a short pre-period, or one that misses a seasonal peak the test falls in, can produce a forecast that is wrong for reasons unrelated to the treatment. And aggregation hides imbalance: one large market in the treated group can dominate the aggregate, so a local disruption there reads as a campaign effect.

None of this is exotic, and that is the method's main strength. The model is a slope and an intercept, its assumptions fit on one line, and its pre-period fit can be plotted and argued over by people who are not statisticians. It is the analysis behind Meridian GeoX, and in a randomised design — where assignment rather than the model is what makes the groups comparable — that simplicity costs very little.

The formula

The pre-period fit, the counterfactual it produces, and the two quantities read off the test window.

The pre-period fit
Ȳ_treat,t = α + β · Ȳ_control,t + εₜ

Ordinary least squares on the aggregated daily or weekly series. Meridian GeoX constrains β ≥ 0.

The counterfactual
Ŷ_treat,t = α̂ + β̂ · Ȳ_control,t through the test window

Driven by what the controls actually did, so shocks common to both groups cancel.

The lift
Δ = Σ ( Ȳ_treat,t − Ŷ_treat,t ) over the test window

Accumulated over the window and scaled to the treated markets' total. A go-dark test reverses the sign.

The interval
widens with the length of the test window

Each day's forecast error accumulates. A placebo run on a quiet window is the cheapest check that the interval is honest.

Worked example

A subscription brand runs a five-week heavy-up of online video in 30 randomly assigned treatment DMAs, with 30 control DMAs, and reads the result with time-based regression on daily sign-ups.

Pre-period
16 weeks of daily data
Pre-period fit
β = 0.97, R² = 0.91
Treated sign-ups over the test window
52,400
Counterfactual forecast
49,200
Estimated lift
+3,200 (+6.5%), 90% interval +1,100 to +5,300
Placebo on a quiet five-week window
−180, interval spans zero
Incremental spend
$240,000

The heavy-up added about 3,200 sign-ups, a 6.5% lift, at an incremental cost of $75 each, and the placebo window showed nothing.

The slope of 0.97 says the treated markets moved almost one for one with the controls before the test, which is what a randomised split should produce and means the counterfactual is close to a straight copy of the control trend. The placebo row is what makes the headline believable: on five weeks where nothing changed, the same machinery found nothing, so it is not manufacturing effects out of weekly noise. The number to act on is the interval rather than the point estimate. Dividing the $240,000 by the two ends of it puts the incremental CPA anywhere between about $45 and $218 — a range wide enough to justify scaling the channel if the target is $250, and nowhere near wide enough to call it if the target is $80. A longer window or more markets would narrow it; reading the $75 as if it were exact would not.

Common misconceptions

“A high pre-period R-squared means the lift estimate is reliable.”
Fit measures how well the controls tracked the treated markets in the past, which is necessary and not sufficient. A contaminated control market tracks especially well, because it is partly treated, and a pre-period that misses the season the test runs in can fit beautifully and forecast badly. A placebo run on a quiet window tests the forecast; R-squared only tests the fit.
“Time-based regression models how the campaign works.”
It models the counterfactual, not the campaign. It says how much the treated markets moved relative to forecast during one window at one spend level, and nothing about carryover after the window, diminishing returns or how the effect would change at another budget. Those are questions for a marketing mix model, which is why a TBR result is most useful as an input to one.

Frequently asked questions

How is time-based regression different from difference-in-differences?
Difference-in-differences assumes the gap between treated and control markets would have stayed constant, so it subtracts the pre-period gap from the test-period gap. Time-based regression fits a slope as well as an intercept, so the treated series can scale with the controls rather than run parallel to them. When the two groups are of similar size and trend the answers are close; when they differ in scale, the slope matters.
How long a pre-period does time-based regression need?
Long enough to contain the patterns the test window will contain. Several weeks of daily data is a practical floor for a stable slope, and if the test falls in a seasonal peak the pre-period needs to have seen comparable weeks, or the fitted relationship will not hold. Much longer is not automatically better, because an old relationship between markets may no longer describe the current one.
Does time-based regression work with only a few markets?
Mechanically yes, because it aggregates them into two series before fitting anything. The risk is in what the aggregate hides: with few markets, one large one can dominate the treated total, and randomisation is less likely to produce balanced groups. Look at each market's share of the aggregate, and treat a result driven by one market with suspicion.

Related terms

  • Counterfactual forecast

    The dotted line on every geo chart — a prediction, not an observation, and the whole result rests on it.

  • Difference-in-differences

    Subtract the untreated group's change from the treated group's — and everything rests on parallel trends.

  • Meridian GeoX

    Google's geo-testing library for Meridian — design, analysis and calibration in one package, and all three still need supervising.

  • Regression analysis

    Fit a line through the data — and the phrase 'holding everything else fixed' is where the trouble starts.

  • Synthetic control

    Build the comparison group instead of finding one — the method for when you have one treated unit.

Calculate it

  • Correlation test

    Pearson r or Spearman rho, with the Fisher-z interval that says how little a small sample knows.

  • Paired t-test

    Before-and-after or matched pairs — size the study on the difference SD, then test it.

Knowing the term is the easy part

Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.

References