Cohort analysis that changes the roadmap

Nearly every quarterly review deck we are shown contains a cohort chart. The triangular grid of months, shaded from dark to light, with retention decaying left to right. Everyone nods at it. Nobody has ever changed a plan because of it.

The chart is not the problem. Cohort analysis is the only routine way to see whether a programme is changing customer behaviour, as opposed to whether a good month happened. The problem is that the standard chart is built to be shown rather than to answer a question, so it answers none.

A cohort view earns its place when somebody could act differently depending on what it says. That constraint changes how you define the cohort, what you plot, and which comparisons you allow yourself to make.

Define the cohort around a decision you might take

Start from the decision, then build the cohort that informs it. The default of grouping by signup month is fine for a health check and almost useless for a roadmap choice.

Useful cohort definitions we come back to:

• First-purchase category or product. Tells you whether some entry points produce better customers, which changes merchandising and acquisition briefs.
• Acquisition channel or campaign. Tells you whether cheap traffic is expensive later, which changes the budget split.
• First-order discount depth, banded. Tells you what you are buying with the welcome offer.
• Onboarding treatment received. Tells you whether the journey you built is doing anything after the first month.
• First-week behaviour, such as completing a profile or a second session. Tells you which activation milestone is worth designing around.

Each of those maps to a decision with a budget behind it. Signup month maps to nothing you can change.

Write the decision on the chart before you build it. One line at the top: "if early-discount cohorts retain materially worse, we cut the welcome offer depth in Q3". A chart with that sentence attached gets discussed differently from one captioned "retention by cohort".

Read the curve in three parts

A retention curve is not one number, and averaging it destroys the information. Read it as three separate things, because each has a different cause and a different fix.

The first drop. From period zero to period one, usually the steepest fall on the chart. This is an onboarding and expectation problem: the product or service did not do what the purchase implied, or nobody helped the customer reach the point where it would. CRM can move this, and it is normally where the fastest wins are.

The slope. The decay across the middle periods. This is habit and relevance. Frequency, assortment, and whether your contact strategy matches the natural purchase cycle. Moves slowly, responds to programme design rather than single campaigns.

The floor. Whether the curve flattens and at what level. A curve that flattens has a loyal core, and the floor height multiplied by cohort size is roughly what your base is worth. A curve that keeps declining has no core, which is a business-model finding rather than a marketing one.

Plot the three separately over successive cohorts. A programme change that improves the first drop but leaves the slope untouched is still a real win, and it will be invisible in a blended churn rate that mixes all three together.

Compare cohorts only when one thing differs

The most common analytical error we see is comparing cohorts that differ in several ways at once and attributing the gap to the most recent change.

The January cohort was acquired during a sale, in a different channel mix, at a different discount depth, and it also got the new onboarding journey. Its retention is better. The onboarding journey gets the credit, the team invests further, and nothing improves next quarter.

Guardrails worth holding to:

1. Hold the acquisition source constant, or split the comparison by source and check the direction is consistent within each.
2. Hold the discount band constant. Promotional cohorts behave differently for at least a year.
3. Compare equal maturity windows. A three-month-old cohort read at month three against a year-old cohort read at month twelve is not a comparison.
4. Check seasonality by looking at the same month in previous years, not only at adjacent months.
5. Watch the denominator. If acquisition volume doubled, the newer cohort contains different people, and mix explains more than treatment.

When too many things differ, say so and stop. An honest "we cannot separate these effects" is a finding, and it usually justifies running an actual experiment next quarter instead of another retrospective read.

Turn the finding into work somebody owns

A cohort finding becomes a roadmap item only if it survives three questions. What would we do differently? How much is it worth? Who owns it?

Work the value question with the same arithmetic every time. Take the difference in repeat purchase rate between the cohorts, apply it to the volume of customers entering that state each month, and convert to contribution rather than revenue. Express the result as annual contribution and state the assumptions beside it.

That converts "discount-acquired customers retain worse" into something a planning meeting can act on. Shifting the welcome offer from twenty per cent to fifteen is worth roughly this much a year if the difference holds, and costs us this many first orders. Now two people can disagree about a number rather than about a chart.

Record the decision and its expected effect where the next reader will find it. The decision log exists for exactly this: cohort findings mature over months, and by the time the evidence arrives the people involved have moved on or reinterpreted what they meant.

Then set the read date. A cohort-driven change takes at least one full cohort maturity window to evaluate, so book the review at the point the data will exist, not at the end of the quarter.

What a cohort chart cannot tell you

Three limits, all worth stating out loud when the chart goes up.

Cohorts are observational. They show that a group behaved differently, never that your treatment caused it. Where a decision is expensive or reversible only at cost, the cohort view is the hypothesis and a holdout or a proper experiment is the evidence. Treating a cohort difference as causal is how a plausible finding becomes a wasted year.

Cohorts also see only the customers you already have. They cannot tell you what the brand campaign did to acquisition quality, or how much of the change came from pricing, distribution or a competitor's move. That question needs the top-down view, which is why we paired it with marketing mix modelling for CRM teams earlier this month. The two methods answer different questions and disagree productively.

The last limit is about lifetime value. A cohort's value estimate is a projection until the cohort has matured, and the projection is most confident exactly when it is least useful. Keep the horizon short, state it, and resist the temptation to extrapolate a curve you have only seen the first third of.

What to do this week

• Take your standing cohort chart and write the decision it would inform at the top. If you cannot, rebuild the cohort definition around one you actually face.
• Split your current retention curve into first drop, slope and floor, and track the three separately across the last six cohorts.
• Pick one cohort difference you already believe in, cost it in annual contribution with the assumptions written down, and take it to the next planning meeting as a proposal rather than an observation.