Bell Statistics

Cause and effect

Methods for answering whether one thing caused another when you cannot randomise, and the biases that make an observed difference look like a causal one.

7 terms

A randomised experiment answers a causal question by construction: assignment is independent of everything else, so a difference in outcomes has nowhere else to come from. When randomisation is impossible — because the change is already live, or applies to whole markets, or is a decision somebody else made — the causal claim has to be built rather than assumed, and the terms here are how.

What every method in this group is doing is constructing a counterfactual: an estimate of what would have happened without the change. Difference-in-differences builds it from a parallel group's trend, synthetic control from a weighted combination of untreated units, and propensity score matching from comparable individuals. They differ in what they assume, and the assumption is always the load-bearing part.

The failure mode they exist to avoid has two names here. Confounding is a third variable driving both the treatment and the outcome; selection bias is the treated group differing from the untreated one in a way that was never measured. Both produce a clean correlation and a wrong causal claim, and neither is detectable from the data alone — which is why the design matters more than the model.

This is most of what we are hired for. See our causal inference work.

Terms in this group

  • Causal inference

    Estimating what an action caused by reconstructing what would have happened without it.

  • Confounding variable

    A common cause of both variables — the reason a strong, stable correlation can mean nothing.

  • Difference-in-differences

    Subtract the untreated group's change from the treated group's — and everything rests on parallel trends.

  • Geo experiment

    Randomise regions instead of users — the way to test marketing that cannot be hidden from a person.

  • Propensity score matching

    Pair like with like on the probability of being treated — and hope nothing important went unmeasured.

  • Selection bias

    When who ends up in the data is not who you meant to study — and more data makes it worse.

  • Synthetic control

    Build the comparison group instead of finding one — the method for when you have one treated unit.

Causal Inference Analysis at Bell Statistics

We build causal measurement for teams that cannot randomise — geo tests, synthetic controls and the models that calibrate against them. See how we work.