What is a hangover effect?

Change a workflow people use every day and they get slower at it before they get faster. Muscle memory points at where the button used to be, a familiar sequence no longer works, and for a period the experience is genuinely worse — not because the new design is worse, but because it is unfamiliar. Measure during that period and a good change reports a loss. This is the hangover effect, and it is the novelty effect with the sign reversed.

It matters most for changes to habitual surfaces and for professional tools, where users have invested real practice in the current arrangement. A consumer browsing a shop casually has little to unlearn; an operations team that runs the same six-step process forty times a day has a great deal. That is why B2B and internal-tool experiments so often show an early regression that recovers, and why shipping decisions in those contexts should almost never be made on a first week.

There is a second, less obvious meaning of the term that matters more for programme hygiene. Effects can persist after an experiment ends: a user exposed to variant B for three weeks has learned its layout, and when they are returned to control for the next experiment they carry that learning with them. Their behaviour in the new test is contaminated by the old one. This is the reason experimentation platforms re-randomise between experiments rather than keeping stable buckets, and it is why running back-to-back tests on the same surface with the same users deserves a washout period.

Separating hangover from genuine harm uses the same tools as novelty. Plot the effect by period rather than cumulatively: a hangover starts negative and climbs towards zero or beyond, while a genuinely worse variant stays down. Segment by prior exposure: new users have nothing to unlearn, so if the loss is concentrated among long-tenured users and absent among newcomers, that is the diagnosis. And if the two eventually converge, the converged value is the durable effect.

The uncomfortable part is that the correct response is often to ship something that currently measures worse. A change with a two-week adaptation cost and a permanent 3% gain is worth making, and any process that requires every experiment to show an immediate win will systematically reject exactly that class of improvement. Deciding in advance how long adaptation should take, and reading the effect after it, is what keeps the discipline honest without making it blind.

A logistics company redesigns the dispatch console its operators use all day. The change removes three clicks from the core task. It is tested for five weeks on tasks completed per hour, and the team segments by whether the operator joined before or after the test began.

The last row is what makes the diagnosis certain rather than hopeful. Operators who joined after the test started showed a 5.4% gain in week one — they had no old console to unlearn, so they experienced the durable effect immediately. That single comparison separates "our users need two weeks to adapt" from "this design is worse", and it is available in week one, long before the time series has resolved. The economics are worth stating too: two weeks at roughly −5% against a permanent +6% pays back in under a month and then compounds. Any process that required week one to be positive would have rejected it. The generalisable rule for tools like this is to decide the adaptation window in advance from how often the task is performed — a daily workflow adapts in days, a monthly one takes months — and to read the effect after that window rather than across it.