Content analytics: which content keeps members

Most content reports answer one question: what was popular? They rank lessons, articles and videos by views, and it is tempting to make more of whatever sits at the top. But for a membership, popularity is not the goal. Retention is. The content that matters most is the content that keeps members subscribed, and that is often not the content with the most views.

A piece that everyone opens once and forgets does less for your business than a quieter piece that members come back to every week. Content analytics for a membership site means looking past views to the link between what members use and whether they stay.

Popular is not the same as valuable

Views are shaped by things that have little to do with value: where a piece appears in your navigation, whether it was in a recent email, how intriguing its title is. A lesson linked from the welcome email will always look popular, simply because every new member sees the link.

Consider a hypothetical guitar lessons membership. Its most viewed page is a gear guide that attracts plenty of search visitors. But members who read it are no more likely to stay than anyone else. Meanwhile, a plain series of weekly practice routines gets modest views, yet members who use it seem to stay much longer. Judging by views alone, the owner would invest in more gear guides.

Questions to ask of your content data

• Which content do members return to repeatedly, not just open once?
• Which content do new members use in their first weeks, and do those members stay longer?
• Which content do members start but not finish?
• Which content has almost no use at all?
• What are members searching for on your site that returns nothing?

These questions build on the broader engagement metrics you track for the membership as a whole, but they zoom in on individual pieces and series.

A simple method: users versus non-users

The core technique is a comparison. Pick a piece or series of content, then split members into two groups: those who used it within a set period and those who did not. Then compare how many in each group were still members some months later.

1. Choose members who all joined within the same window, such as the same quarter, so they had similar amounts of time to use the content.
2. Decide what counts as using the content: opening it, completing it, or returning to it more than once.
3. Split the group into users and non-users.
4. Check how many in each group are still paying after, say, six months.

Comparing members who joined at the same time is important. If you mix long-standing members with new ones, you end up measuring tenure, not content. If this sounds familiar, it is a close cousin of cohort analysis.

A worked example

The guitar membership looks at 400 members who joined over one quarter. Illustrative figures:

• 150 completed at least three of the weekly practice routines in their first month. After six months, 105 of them were still members: seven in every ten.
• 250 did not. After six months, 100 of them were still members: four in every ten.

That is a large gap. It is tempting to conclude that the practice routines cause members to stay, and to push every new member toward them.

The correlation trap, and how to test your idea

Here is the catch. Highly motivated members tend to use more of everything. They might have stayed anyway, routines or not. The gap may partly reflect who the members are, not what the content does.

You can test this cheaply. Make the routines more prominent for new members: feature them in the welcome email, on the member home page and in the first-week check-in. Then compare cohorts before and after the change. If more new members use the routines and retention for those cohorts rises, you have much stronger evidence that the content itself matters. If usage rises but retention does not, the routines were a sign of motivation, not a cause of staying.

It also helps to run the same comparison on other content. If members who used almost anything stayed longer, you are mainly seeing motivation. If one or two pieces stand out well above the rest, those are your real keepers.

Other signals worth watching

• Completion: a video that most members abandon early may be too long or start too slowly.
• Repeat use: content members return to, such as templates, checklists and reference guides, often does more for retention than content consumed once.
• Empty searches: searches on your site that return nothing are a ready-made list of content members want and cannot find.
• Silent content: pieces with almost no use for months are candidates to improve, combine or remove, as part of organizing a growing content library.

Turning findings into content decisions

1. List your ten most-used and ten least-used pieces, and note which ones members return to.
2. Run the users-versus-non-users comparison for your three or four most important series.
3. For any piece that seems linked to retention, make it easier to find, especially for new members, and watch the next few cohorts.
4. Plan new content around the topics and formats that keep members, not just the ones that draw clicks.
5. Review the findings each quarter, since what members value can shift as your membership grows and changes.