Break your numbers down by plan and source

Your monthly churn figure, your average length of membership and your trial conversion rate each summarize the whole membership in one number. Each of those figures is an average, and an average describes a membership that does not quite exist. Behind it sit groups of members who behave very differently: monthly and annual members, people who found you through search and people who arrived through a partner's discount, members of your entry level and members of your top level.

Breaking your numbers down, often called segmenting, means calculating the same figure separately for each group. It is the single most reliable way to turn a vague worry into a specific problem you can fix.

Choose a few dimensions that matter

A dimension is a way of dividing members into groups. You could slice your data in dozens of ways, but most insight comes from a handful:

• Plan or level: entry, standard, premium.
• Billing period: monthly, quarterly, annual.
• Acquisition source: search, social media, referrals, partners, paid ads, your email list.
• Price paid at joining: full price or promotional discount.
• Join period: the month or quarter members signed up, which lets you compare newer members with older ones.

Start with plan and source. They are the two dimensions most likely to change a decision, because they connect directly to what you sell and where you spend your marketing effort.

Which figures to break down

Not every number needs splitting. The ones that repay the effort are:

• New members by source, so you know where growth comes from.
• Churn by plan, billing period and source, so you know who leaves.
• Revenue per member by source, since some channels bring people who choose cheaper plans; see average revenue per member.
• Early engagement by source, because it hints at future churn before cancellations arrive.

For churn, calculate both member and revenue versions by plan, since revenue churn and member churn can tell different stories when levels behave differently.

A worked example

Take a hypothetical leathercraft membership run by a maker named Femi. The figures are invented to show the method. At the start of a month he has 1,000 paying members, and 62 cancel. Overall churn is 62 ÷ 1,000, or 6.2%. On its own, that figure suggests a general problem with the membership.

Split by billing period:

• 700 monthly members, 56 cancelled: 8%.
• 300 annual members, on a monthly-equivalent basis 6 cancelled: 2%.

Split by source:

• 400 members from search, 16 cancelled: 4%.
• 300 members from social media, 21 cancelled: 7%.
• 300 members from a partner promotion with a half-price first quarter, 25 cancelled: about 8.3%.

Now the picture is clearer. Members who arrive through search stay well. The partner promotion brings in members who leave at twice the rate. Monthly members leave at four times the rate of annual members. Femi now has two specific projects instead of one vague worry: rethinking the promotion and making the annual plan more visible.

Beware the mix trap

Breaking numbers down also protects you from a trap that catches many owners: the overall figure can move in the opposite direction from every group inside it.

Suppose in one month Femi has 600 search members, of whom 24 cancel (4%), and 400 promotion members, of whom 40 cancel (10%). Overall churn is 64 ÷ 1,000, or 6.4%.

A few months later, after a large promotion, he has 400 search members, of whom 14 cancel (3.5%), and 600 promotion members, of whom 54 cancel (9%). Overall churn is 68 ÷ 1,000, or 6.8%.

Churn improved in both groups, yet the overall figure got worse, simply because the mix shifted toward the group that always leaves faster. Without the breakdown, Femi might conclude that his retention work had failed. In fact it worked; his marketing had changed who was joining.

Keep your segments honest

• Watch the group sizes. If a segment has only a dozen members, one cancellation moves its churn by several points. Combine several months, or merge small groups into an Other category.
• Record the source at sign-up. You cannot split by source later if you did not capture it at the time. Consistent tagged links and a short how-did-you-hear question make this possible.
• Use the same definitions for every group. If you exclude trials from churn overall, exclude them in every segment.
• Stop at two dimensions at once. Plan by source is useful. Plan by source by region by device produces tiny groups and false patterns.
• Look for differences big enough to act on. A gap of a fraction of a point between two groups is rarely worth a project. A gap of double is.

Setting up your first breakdown

1. Export your members with plan, billing period, source, join date and status.
2. Calculate this month's churn and new members for the whole membership.
3. Recalculate both by plan, then by source.
4. Note the group sizes beside each figure, and flag any group too small to trust.
5. Look for the largest gap between groups and write down one possible reason for it.
6. Choose one action aimed at that group, and check the same breakdown again in a month or two.