Five Common Blind Spots in Marketing Data
In today’s privacy-focused era, the different attribution models create many blind spots for marketing analysts and decision makers. However MMM & Geo Tests can help.
What is correlation?
Correlation measures how strongly two variables move together, on a scale from minus one to plus one. It describes the direction and tightness of a straight-line relationship, and says nothing at all about which variable causes which.
r
In plain English
Correlation puts a number on co-movement. Plus one means the two variables rise together in perfect lockstep; minus one means one rises exactly as the other falls; zero means no straight-line relationship at all. Pearson's r is the standard measure and is a covariance rescaled by the two variables' own standard deviations, which is what makes it unitless and bounded — you can correlate pounds with seconds and get a number between −1 and 1.
That rescaling has a consequence people rarely account for: r is not comparable across samples with different spread. Two markets with an identical underlying relationship between spend and revenue will report different correlations if one has more variation in spend, because the denominator differs. When the comparison actually matters, compare regression slopes in original units instead — the slope answers "how much does revenue move per pound" and the correlation answers "how tightly do they track", and only the first is stable.
"Correlation is not causation" is true and nearly useless, because it offers no guidance on the number in front of you. Three more actionable failures are worth knowing. Attenuation: measurement error in either variable drags r towards zero, and the observable ceiling is the square root of the product of the two measures' reliabilities — so 0.45 between two noisy proxies can reflect something much stronger. Restriction of range: filtering the sample on either variable shrinks r, which happens invisibly when you analyse only active accounts or only campaigns above a spend floor. And non-linearity: a clean inverted U — the shape diminishing returns actually takes — can produce an r near zero while the relationship is strong and obvious on a scatter plot.
Time series are where correlation does the most damage in practice. Two unrelated series that both trend upwards will correlate above 0.9 as a matter of arithmetic, and the significance test makes it worse: weekly observations are autocorrelated, so 104 weeks contain far fewer than 104 independent observations and the confidence interval comes out far too narrow. Difference or deseasonalise both series before correlating them, and treat any correlation between two raw trending series as meaningless until you have.
Read the interval before the coefficient. An r of 0.42 from 30 observations carries a 95% interval of roughly 0.07 to 0.68 — technically significant, and consistent both with a relationship too weak to matter and with one strong enough to plan around. Small-sample correlations are among the least stable statistics in routine use, and the interval is the only thing on the output that says so. If the question is causal rather than descriptive, no sample size helps: that needs a geo experiment, randomisation, or an explicit causal model.
The formula
A covariance divided by two standard deviations, plus the transformation that makes an honest interval possible.
- Pearson correlation
r = Σ(xᵢ − x̄)(yᵢ − ȳ) / √( Σ(xᵢ − x̄)² · Σ(yᵢ − ȳ)² )The denominator is why r is not comparable across samples with different spread — the same slope gives a different r in a more variable market.
- Spearman's rho
ρ_s = Pearson r computed on the ranks of x and yHandles monotone-but-curved relationships and resists outliers. The familiar 1 − 6Σd²/(n(n²−1)) shortcut is correct only with no ties.
- Fisher's z interval
z = ½·ln((1+r)/(1−r)), SE(z) = 1/√(n − 3)The interval is built on this scale and mapped back with tanh, because r's own distribution is skewed whenever the true value is not zero — see the correlation test calculator.
- Attenuation ceiling
r_observed ≈ r_true · √( reliability_x · reliability_y )Two measures with reliability 0.8 each cap the observable correlation at 0.8, however many rows you collect.
Worked example
An analyst reports that weekly paid-search spend and weekly revenue correlate at r = 0.87 over two years, and concludes that search is the strongest driver of revenue in the business.
The 0.87 is almost entirely shared trend and seasonality. Once both are removed, week-to-week movements in spend and revenue correlate at 0.12.
Both series grew over two years and both peak in the same weeks, which is enough to manufacture 0.87 with no relationship whatsoever between them. The naive interval of 0.81 to 0.91 makes it look nailed down, and it is built on 104 observations that contain roughly 26 observations' worth of independent information — so it is about twice as narrow as it should be even before the trend problem. The residual 0.12 is the honest descriptive figure, and it is still not causal: spend rises when demand rises, so what remains is confounded too. The geo test is the only number here that supports a budget decision, which is the general lesson — correlation is a useful diagnostic and a poor foundation.
Common misconceptions
- “A correlation near zero means there is no relationship.”
- It means no straight-line relationship. An inverted U, a threshold effect or a saturation curve can all produce r near zero while being strong and obvious on a scatter plot — and saturation is exactly the shape marketing response takes. Plot the data before concluding anything from a small r.
- “r-squared tells us what share of revenue this variable explains.”
- It is the share of *variance* linearly associated with the other variable, in this sample, and it is biased upward as an estimate of the population value. It is not a share of revenue and it is not a causal contribution. It is also a sobering number: r = 0.30 corresponds to 9% of shared variance.
- “Our correlation is highly significant, so it is a reliable finding.”
- Significance here only rejects the hypothesis that the true correlation is exactly zero, which is rarely plausible for two business metrics anyway. With a few thousand rows almost any r clears the threshold. The interval is the informative object, and on time-series data the reported one is usually far too narrow because the observations are not independent.
Frequently asked questions
What counts as a strong correlation?
It depends entirely on the domain, and the conventional labels travel badly. In physical measurement, 0.9 is unremarkable; in behavioural data, 0.4 can be a substantial finding. More useful than a benchmark is squaring it: r = 0.30 is 9% of shared variance and r = 0.50 is 25%, which is usually more sobering than the coefficient itself sounds. Judge it against what you would need for the decision at hand.
When should I use Spearman instead of Pearson?
When either variable is ordinal, when the relationship is monotone but curved, or when a handful of extreme points would otherwise dominate the answer. If the two coefficients differ substantially, that gap is itself informative — it usually indicates non-linearity or influential outliers, and you should find out which before reporting either number.
How many observations do I need to measure a correlation?
Far more than most analyses use. At 80% power and the conventional 5% two-sided level you need about 29 observations to detect a correlation of 0.5, 85 for 0.3, and 782 for 0.1. Correlations are routinely reported from twenty or thirty rows, where the interval is so wide that the estimate carries almost no information — an observed 0.35 from 25 observations spans roughly −0.05 to 0.66.
Calculate it
Knowing the term is the easy part
Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.