Beta-test your course before the full release

You have spent weeks building a course. You know the material inside out, every lesson makes sense to you and the activities seem clear. That is exactly the problem. You cannot see the course the way a new learner sees it, because you already know what every lesson means.

A beta test puts the course in front of a small group of real learners before the full release. They find the confusing explanations, the missing steps, the activities that take three times as long as you promised and the broken links. Fixing those problems with ten testers is far easier, and far less embarrassing, than fixing them after hundreds of members have hit them.

What a course beta test is for

A course beta test is not about whether people want the course. That question should be settled earlier, and running a pilot program before you build the full membership is one way to settle it. A beta tests whether the course works. It answers questions such as:

• Can learners get through every lesson without getting stuck?
• Where are explanations unclear or missing a step?
• How long do lessons and activities really take?
• Do the activities, quizzes and downloads work as intended?
• Do learners reach the result the course promises?
• What technical problems appear on different devices?

Write your own list of questions before the beta begins. It keeps you focused on what you need to learn.

Recruit the right testers

The best testers look like the learners the course is designed for. Friends, family and fellow experts are tempting because they are easy to ask, but they tend to be too kind or too knowledgeable. A small group, perhaps eight to fifteen people, is usually enough to reveal the main problems. Aim for a mix:

• A few complete beginners, if the course is for beginners.
• A few people at the upper edge of your audience.
• People who use different devices, especially phones.
• At least one or two people who are not already enthusiastic fans.

Offer testers something fair in return, such as free access to the finished course, a discount or a credit in the course. Be clear about what you are asking. Imagine a hypothetical membership for new backyard chicken keepers, run by a small-farm owner named Winifred. Her invitation read:

I'm looking for ten members to test my new course, Your First Flock, before it opens to everyone. Over three weeks, you'd work through the lessons, answer one quick question at the end of each and join a 20-minute call at the end. You'd get the finished course free, and your feedback will shape it. I'm especially keen to hear from people who haven't kept chickens yet. Reply to this email if you're interested.

Collect feedback while testers learn

Feedback gathered at the end of a beta is often vague, because testers have forgotten the details. Collect it as they go:

• A one-question check after each lesson. “Was anything unclear, missing or harder than it needed to be?”
• A time log. Ask testers to note roughly how long each lesson and activity took them.
• A confusion thread. A private discussion space where testers can post questions as they arise. Every question is a sign that a lesson needs work.
• Watching a few testers learn. Ask two or three testers to share their screen and think out loud as they work through a lesson, while you watch without helping. You will see hesitations, wrong clicks and skipped instructions that nobody would think to report.
• A short end-of-beta call to ask what worked, what did not and whether they got the promised result.

Run the beta on a schedule

An open-ended beta drifts. Give it a start date, an end date and a weekly rhythm. Winifred ran hers over three weeks: a short welcome call to explain what she needed, a weekly email with the lessons to cover and a reminder to answer the lesson questions, and a final call. She checked the confusion thread daily and fixed small problems, such as broken links or a missing download, straight away.

For larger problems, resist the urge to rewrite lessons during the beta. Note them, keep going and fix them properly afterwards, so all testers experience the same version.

Sort what you learn and act on it

When the beta ends, gather everything in one place and sort it into three groups:

• Must fix. Anything that stopped testers progressing, caused confusion for several people or was simply wrong.
• Should fix. Problems several testers mentioned that slowed them down but did not stop them.
• Consider. Single suggestions and preferences. Note them, but do not redesign the course around one person's opinion.

Winifred's must-fix list included a lesson on building a coop that assumed woodworking skills most testers did not have, and a feeding activity that took most testers over an hour instead of the promised twenty minutes. She added a simpler coop option and split the feeding activity in two. Update time estimates across the course using the real times from the log.

Finally, thank your testers, tell them what changed because of their feedback and ask whether you can quote any of their comments. A tester who reached the result is often your first and most credible testimonial.

Planning your course beta

1. Write the questions you need the beta to answer.
2. Recruit a small group of testers who match your target learners, including beginners and phone users.
3. Decide what testers receive and exactly what you are asking of them.
4. Set up a one-question lesson check, a time log and a confusion thread.
5. Arrange to watch two or three testers work through a lesson.
6. Run the beta on a fixed schedule with a weekly email and a closing call.
7. Sort feedback into must fix, should fix and consider, and fix the first group before launch.
8. Thank testers, tell them what changed and ask for permission to share their results.