Reading cancellation reasons with AI

Every cancellation comes with a lesson, but most of those lessons go unread. Exit answers pile up in a form tool, reasons are buried in support emails, and the occasional honest conversation lives only in your memory. When you finally look, you see a blur of “too expensive,” “no time” and “other,” and it is hard to know what to fix.

AI is well suited to this job. It can read hundreds of short, messy answers and sort them into themes far faster than you can. The value comes from how you prepare the data, how you instruct the AI and how carefully you check its conclusions. This guide covers all three, so your exit feedback becomes a short list of changes you can act on.

Gather exit feedback in one place

Cancellation reasons usually come from several sources. Pull them together into one spreadsheet with one row per cancellation:

• Answers from your cancellation form or exit survey
• Cancellation emails and support tickets
• Notes from any calls or conversations with leaving members
• Replies to win-back or check-in messages that explain why someone left

Add a few useful columns alongside the reason: membership level, how many months they were a member, and the month they cancelled. These let you slice the results later.

Clean and anonymize before any AI sees it

Exit feedback often contains personal details. Members mention job losses, illness, family changes and names of other people. Before you use an AI tool, remove names, email addresses and member numbers, and edit out any sensitive details in the free text that are not needed to understand the reason. Replace each row's member with a code.

Use a paid business plan and check that the data and training settings keep your uploads out of model training. Read the vendor's terms if you are unsure. The underlying principle is simple: the AI needs to know why people leave, not who they are.

Code the reasons with your own categories

“Coding” is a research term for labeling each answer with one or more categories. It works best when you provide a starting set of categories rather than letting the AI invent a different set every time. A reasonable starting list might include: price or budget, lack of time, content not relevant, content too basic or too advanced, technical problems, poor support experience, achieved their goal, life circumstances, and unclear.

Then ask an assistant such as ChatGPT, Claude, Gemini or Microsoft Copilot to apply them:

This file contains anonymized cancellation reasons from members of [describe your membership]. Label each row with one main reason and, if clearly present, one secondary reason, using only these categories: [paste your categories]. If a reason fits none of them, label it Other and suggest a new category name. Add a column called Underlying, where you note in a few words any deeper cause hinted at in the text, for example “price, but mentions never using live calls.” Then give me a summary table showing how many rows fall in each main category, with two exact quotes for each. Do not change or invent quotes.

Consider a hypothetical online pottery membership called Kiln and Clay. Its owner, Rosalind, found that the largest category was price, as she expected. The Underlying column told a different story: many of those members mentioned they had never used the live critique sessions, which were the main thing the higher price paid for.

Look beneath “too expensive”

Price is often the easiest reason to give, not the real one. A member who is getting great value rarely leaves over a modest fee. When price dominates, ask the AI to separate price complaints that mention something else, such as unused features, lack of time or confusion about what was included. Those are value problems, and they are much more fixable than a budget problem.

Slice by tenure and level

Reasons for leaving change over a member's lifetime. Ask the AI to break the coded results down by how long members stayed and by membership level:

• Members who leave in the first month often point to onboarding or expectation gaps.
• Members who leave after six months or more may have achieved their goal or run out of new material.
• Members on a higher level may cite underused premium features.

Check any counts in a spreadsheet yourself, since AI can miscount. Compare the patterns with what your activity data shows; if you have been spotting members at risk of leaving, the two views should support each other.

Check the results before you act

Treat the AI's coding as a first pass, not a finding. Pick twenty rows at random and code them yourself without looking at the AI's labels, then compare. Where you disagree, adjust your category definitions and rerun. Search for a few of the quotes to confirm they are exact. Read the Other group carefully, because new problems appear there first. If your exit questions themselves are vague, it may be worth revisiting them using the approach in member surveys with AI.

Turn themes into three fixes

A long list of reasons is not a plan. Pick the three themes that are both common and within your control, and decide one concrete change for each. Rosalind decided to feature the live critiques in onboarding, add a reminder before each session and offer a lower level without critiques. For the non-AI fundamentals of matching fixes to reasons, see why members cancel, and what to do about each reason.

Repeat the analysis every few months with the same categories, so you can see whether your fixes are shrinking the themes they target.

Your first steps

1. Collect every source of exit feedback into one spreadsheet.
2. Add tenure, level and cancellation month columns.
3. Remove names and sensitive details, and replace members with codes.
4. Write your starting category list.
5. Run the coding prompt on a business plan with training turned off.
6. Check twenty rows by hand and verify the quotes.
7. Choose three fixes and schedule your next review.