The Ultimate Guide to Choosing Your Perfect MMM Tool
Robyn, Meridian, Orbit, or custom-made MMM? In this blog post, we compare leading free, paid or custom MMM tools, to help you decide which best fits your needs.
What is multicollinearity?
Multicollinearity is when predictors in a model are strongly correlated with each other. The model as a whole still predicts well, but it cannot tell the correlated variables apart, so their individual coefficients become unstable and easy to misread.
In plain English
A model with two predictors that always move together faces an impossible question. If television and radio spend rise and fall in lockstep across the whole history, no amount of data can say which one drove sales — every combination of coefficients that sums correctly fits equally well. The model does not fail; it produces an answer, and that answer is arbitrary within a wide range. That is multicollinearity, and it is the defining constraint on marketing mix modelling.
The symptoms are recognisable once you know them. Coefficients with implausible magnitudes or the wrong sign. Large standard errors on individual predictors while the model's overall fit is excellent — the classic signature, where the F-test is highly significant and no single t-test is. And extreme instability: dropping one observation, or adding one variable, moves a coefficient by half its value. Any of these on a model with correlated predictors points at the same cause.
What it does not do is bias the estimates or harm prediction. The coefficients remain unbiased on average; they are merely imprecise, and the model's fitted values are unaffected because the collinear variables between them still carry the same information. So the question to ask is what the model is for. If it is forecasting, multicollinearity can usually be ignored. If it is attributing effects to individual channels — which is exactly what a media model is for — it is the central problem.
The standard diagnostic is the variance inflation factor, which says how much wider a coefficient's confidence interval is than it would have been with uncorrelated predictors. A VIF of 10 means the standard error is about three times larger than it would otherwise be. The conventional thresholds of 5 or 10 are rules of thumb rather than tests, and the more useful habit is to look at the correlation matrix among predictors first, since that tells you which variables are the problem rather than merely that there is one.
The fixes are mostly about information rather than technique. Combining collinear variables into a single index removes the ambiguity by declining to answer the question. Regularisation — ridge in particular — stabilises the coefficients by shrinking them, at the cost of introducing bias and making interpretation harder. Bayesian priors do something similar and more transparently, which is one reason modern media models are Bayesian. But the only real fix is variation: flight the channels independently, or run a geo experiment that moves one and not the other, so the data can finally separate them.
The formula
The variance inflation factor is the standard measure, and it comes directly from how much of one predictor the others can already explain.
- Variance inflation factor
VIF_j = 1 / ( 1 − R²_j )R²_j is from regressing predictor j on all the other predictors. VIF = 1 means no collinearity; 5 means the variance is five times inflated.
- Effect on the standard error
SE(β̂_j) ∝ √( VIF_j )VIF of 10 makes the confidence interval about 3.2 times wider. That is the entire practical consequence — the interval, not the estimate.
- Why the estimator breaks down
β̂ = (XᵀX)⁻¹Xᵀy, XᵀX near-singularPerfectly collinear predictors make the matrix non-invertible outright. Near-collinearity makes the inverse enormous, which is where the instability comes from.
- Ridge regression
β̂_ridge = (XᵀX + λI)⁻¹XᵀyAdding λ to the diagonal makes the inverse well-behaved and the coefficients stable, at the price of bias — see the correlation calculator for measuring the collinearity itself.
Worked example
A media model includes television and radio spend, which the media agency has always bought together as a single broadcast package. Over 156 weeks the two series correlate at 0.94. The model is refitted on three overlapping windows to check stability.
The television coefficient swings from 0.9 to 5.1 across overlapping windows. Combined into one broadcast variable, the coefficient is stable at 2.2 with a fifth of the standard error.
Nothing is wrong with the data or the fit — the model predicts sales well in every window. What it cannot do is answer the question it was asked, because television and radio have never moved independently, and the swing across windows is the model reporting that honestly. Splitting a stable £2.20 return into an arbitrary division between two channels and then setting budgets on that division is how a model produces confident nonsense. The right answer for now is the combined variable, and the right answer for next year is to make the agency flight television and radio independently for two quarters, which is the only thing that will let any model separate them.
Common misconceptions
- “Multicollinearity biases the coefficients.”
- It does not — the estimates remain unbiased on average, and prediction is unaffected. What suffers is precision: the standard errors inflate, so any single sample can land far from the truth. That is why the symptom is instability across specifications rather than a consistent error in one direction.
- “All the VIFs are below 5, so collinearity is not a problem.”
- VIF thresholds are conventions, not tests, and what matters is whether your coefficients are precise enough for the decision. With 104 observations and a dozen predictors, a VIF of 4 can still leave an interval too wide to distinguish two channels. Judge the interval against the decision rather than the VIF against a rule of thumb.
- “We should drop one of the correlated variables.”
- That trades imprecision for bias: whatever the dropped variable contributed is now absorbed by the one you kept, so its coefficient is systematically wrong rather than merely uncertain. Combining them into a single index is usually more honest, since it declines to answer a question the data cannot answer instead of answering it incorrectly.
Frequently asked questions
How do I detect multicollinearity?
Start with the correlation matrix among predictors, which tells you which pairs are the problem. Then compute variance inflation factors, treating 5 and 10 as flags rather than verdicts. The most convincing diagnostic is stability: refit the model on overlapping subsets of the data and see whether the coefficients hold. Large swings on a well-fitting model are collinearity almost every time.
Does multicollinearity matter if I only care about prediction?
Barely. The fitted values and predictive accuracy are unaffected, because the correlated predictors jointly carry the same information however the coefficients are divided among them. The one caveat is that a model fitted on collinear data can behave badly if the correlation breaks down in future — if television and radio start moving independently, a model that never learned to tell them apart will extrapolate poorly.
Why is multicollinearity such a problem for marketing mix models?
Because marketing budgets move together by design. Everything ramps for the Christmas season, campaigns launch across channels simultaneously, and budgets are set as proportions of a total that rises and falls as one. That leaves very little independent variation for the model to learn from, on a sample of perhaps 150 weekly observations. It is the main reason to deliberately vary spend and to calibrate the model with geo experiments.
Calculate it
Knowing the term is the easy part
Applying it to a live measurement problem is the part that goes wrong. If you are designing an experiment, reading a result you do not trust, or trying to work out what your marketing actually caused, that is the work we do.