SUMMARY: Getting people to meditate is hard. Getting them to meditate consistently is even harder.
Headspace, one of the world’s leading meditation apps, partnered with Irrational Labs and Purchasely to redesign their onboarding experience for new free trial users, with the goal of increasing active use during the free trial period.
Breaking down this goal, we hypothesized that better onboarding would drive more meditation course starts, more course starts would drive deeper engagement, and deeper engagement would build lasting meditation habits.
The first two links in this chain held. Through application of behavioral science principles, we saw some strong results:
- We more than doubled course starts (from ~31% to ~63%) by adding a short personalization quiz that helped users feel the recommended course was right for them.
- When we asked users to make a concrete plan for when and how they’d meditate, app opens rose 7.5% and unique app open days rose 4%.
But the results also revealed an important limitation: while we got more people to start a course and return to the app, we saw no statistically significant increase in active meditation days. The likely reasons: users couldn’t easily find their course when they came back, and course content didn’t always match what new meditators expected.
The lesson: behavioral design at onboarding can drive early action and engagement, but real habit formation requires behavioral design across the full experience, not just at the front door.
The Challenge
Irrational Labs partnered with Headspace and Purchasely to test changes to Headspace’s onboarding flow to boost active use during users’ free trial period. The Purchasely no-code platform was the essential enabler, allowing the team to rapidly design, deploy, and test five complex, unique experimental variations of the onboarding flow.
Our collaboration aimed to build on our unique strengths to deliver the most impactful intervention to help new Headspace users build a healthy meditation practice:
- Irrational Labs’ strengths in designing and delivering behavior change experiments
- Purchasely’s in-app experience solution to design complex user journeys quickly thanks to its no-code technology
- Headspace’s commitment to improving users’ outcomes using evidence based methods
Goal: Increase active use during the free trial
People face many psychological barriers when they try to form a new habit like meditation:
- Intention-action gap: People often have good intentions to follow through on their promise to themselves to meditate, but struggle due to high perceived effort.
- Effort aversion: While meditation is often portrayed as relaxing, people to new meditation often report finding it hard to “quiet their mind”.
- Choice overload: Headspace offers users many courses, programs, and single guided meditations. While having some choice is often important, having too many choices can make it hard for new users to know where to start.
- Unclear mental models: New users may be unsure about what meditations to do, what courses to take, how long to do it, how many days they should meditate or how many times a day they should meditate.
- Limited attention: People live busy lives, and without frequent reminders, people may simply forget to come back to the app and meditate.
Based on these barriers, we came up with a number of hypotheses on how Headspace could make changes to their product to increase meditation.
Our key hypotheses
1. Tell people what success is and how they are doing.
For new Headspace users, it can be hard to identify what success in using Headspace looks like. How much should a new user meditate or listen to sleep or focus content? Once a day? Three times a day?
While this obviously can vary by person, new users coming to Headspace often have little experience with meditation. They don’t know what success is — they rely on Headspace to help them figure out 1) how to meditate and 2) how much to meditate.
However, in the current app, there isn’t a clear definition of success. If you meditate every other day, is that good? If people don’t know what success looks like they are more likely to fail because they don’t have a clear goal to aim for.
By means of comparison Apple’s “daily rings” feature is so successful because each day users know what success is and how they did. To avoid stressing users out with this streak-like mechanic, Apple gives users control allowing them to turn off this feature in settings if they find it to be too stressful.
2. Make it ridiculously easy in the beginning.
Friction can be a significant barrier to behavior change. If new users are finding the meditations they try too long, or if they struggle to choose a course to start because there are so many options, they might stop meditating entirely. So, an important question we identified was, can people still get the benefits of meditation while meditating for shorter periods of time?
Previous Headspace research shows that people can get benefits from meditation sessions as short as 10 minutes. But could we reduce session length even further and get even more people to meditate? We explored ideas around shortening meditation duration to reduce friction and increase active days but Headspace already offered sessions of 2 and 5 minute durations.
3. Get people to think and plan ahead
A proven recipe in behavior change is “implementation intentions” – making a plan to do something before you do it. If you don’t plan on going for a run today, it won’t happen. Making plans is particularly important when trying something new if you want to actually follow through because automatic routines haven’t taken hold yet.
To decide on which of these theories of change we would build out into designs for testing we reviewed previous experiments and qualitative studies conducted by Headspace. These studies provided useful context. Specifically, we saw that:
- Headspace users with more active days during their free trial are more likely to convert to the paid product
- Previous Headspace experiments showed matching users to a course increased engagement by 20%
- Headspace’s user journal studies showed that users wanted clear next steps and a more tailored experience
Based on these inputs, the team recommended a variety of solutions — including using streaks similar to Apple’s Daily Rings design. Based on the feedback from the Headspace team, this idea was deprioritized and we focused on other potential design solutions.
After multiple design iterations, it was decided that “courses” were the closest thing Headspace had in their current tech stack that could be designed to meet the above solution design principles.
We hypothesized that using a short onboarding quiz to match users to personalized courses would give them a more tailored experience to drive up fit and provide clear next steps for how to continue their meditation.
Our final theory of change: Increased course starts → more engaged users → more meditation.
Results
So, what happened? Did we increase course starts? And does increasing course starts, increase engagement and meditation?
Yes! And no… Okay, let us explain.
Did we increase course starts?
Yes, we were able to increase the number of course starts. And by a large amount.
By using the Purchasely platform to introduce a few simple questions into the onboarding flow, we created idiosyncratic fit — a perception that the experience was unique to the user. This was a massive driver of adoption. Looking at the graph below, it’s easy to see that all of our treatments increased course starts from a baseline of 31.25% in the control group all the way up to 62.97% in our perceived fit condition. Our perceived fit condition actually doubled course starts.
Learning 1: If you want someone to do something, like starting a course, including it in the onboarding flow can drive behavior when users’ motivation is highest.
So far so good.
Our second hypothesis was that just by adding questions, we could increase motivation to sign up for a course. While adding multiple questions to an onboarding flow in-app can typically be technically challenging, Headspace simply added a quiz to their onboarding flow using Purchasely.
We can test this by looking at the difference between our perceived fit condition and the “default to basics” condition because the only difference between these conditions is whether we asked users questions or not before recommending the basics course.
There is a statistically significant increase in course starts between the default to basics condition (no questions) and the perceived fit condition (questions), with course starts being 7.6 percentage points higher in the perceived fit condition where we ask users questions.
💡 Why do we see this increase in course starts between the default to basics and perceived fit condition?
Remember that new users were recommended the exact same course.
Asking users to answer a number of questions about their experience with meditation and plans for using Headspace, creates a perception that the recommended course will be tailored to them even if everyone will be assigned to the basics course anyway. This perception creates idiosyncratic fit, where the user perceives the course as being uniquely beneficial for them as the recommendation has been tailored to them and their needs.
This also raises the question: why don’t we see the same effect in our personalized fit and precommitment conditions, where we actually matched users to a course based on their answers?
One explanation is that matching users to a specific course is hard, and that since these users were new to Headspace, it was hard to actually outperform the best beginner-friendly content Headspace already has, which is the Basics course. Creating more courses designed specifically for beginners with different focuses based on the user’s need state could help to improve idiosyncratic fit while ensuring the recommended content is beginner friendly.
Another explanation could be that we over prioritized sleep as a need state in our recommendation logic. In our logic tree, if a user selected a sleep related response as one of their multiple responses for need state, or in response to the question “what do you want help with”, or in response to “how will you use Headspace?”, we always recommended a sleep course. It’s possible that users said sleep was one of their concerns, but it may have not been their main concern as our logic implied. This is supported by the fact that our data also showed that “stress” and “anxiety” were the most commonly selected need states but we matched users with stress or anxiety related courses at much lower rates.
Learning 2: Adding questions to signal that something is “right for me” can further increase adoption of the key behavior.
Learning 3: It may be hard to outperform the best beginner friendly content — probably because it’s easy to be wrong and harder to be right.
Do courses increase engagement?
We managed to get people to start a course, but were we successful in increasing engagement? Yes.
Our precommitment condition, which asked new users to make a plan about how, when, and where they would meditate increased both the number of days users opened the app and the number of unique days on which they opened the app. Specifically, our precommitment condition increased app opens by 7.5% and app open days by 4%.
💡 Why was our precommitment condition successful in increasing engagement?
We believe this was because it helped people form a clear mental model of how often they should try to meditate and make a plan to follow through on their intention, helping to close the intention-action gap somewhat. For example we asked people to specify which days they would meditate and what triggers they wanted to be associated with using Headspace for them (“In the morning, to start my day” or “Throughout the day instead of scrolling social media”).
Do courses increase meditation?
Let’s take a second to recall our final theory of change.
Our final theory of change: Increased course sign ups → more engagement → more meditation
We managed to increase course starts and increase engagement, so according to our theory of change we should have seen an increase in meditation, but did we? On the whole, no.
Looking at the average number of active days users had on Headspace we saw no statistically significant difference between our control condition and any of our redesigns.
When we used a hurdle model to look at the effect of our treatments on encouraging people to start meditating versus increasing the amount of meditation they engaged in, our precommitment condition slightly increased how often people who meditated at least once continued to meditate but the effect decayed quickly after the first 2-3 days.
💡 How can we explain this set of results?
We successfully increased course starts. We successfully increased how often people came back to Headspace. But we didn’t increase how often new users overall actively meditated. Why?
We have a few hypotheses.
- People couldn’t find their course when they came back: At the time of our experiment, when users re-opened the app, they would land on the today page which would contain not just their course but a range of other meditation options. People may not have been able to identify which option was their course and so abandon before they start a meditation. When designing our interventions we had worried about this and had designed a return modal which would let users jump back into their course with one-click, but the design was cut during implementation.
- The mental model of meditation we set with course content didn’t align with users’ mental model of meditation: New users in the control condition complete a breathing course as the first exercise they complete in Headspace. This likely aligns with inexperienced users expectations of meditation, that it involves sitting still and breathing. Meditation can be, and is, more than that. Course content tends to be more of a guided meditation with a teacher leading you through a longer session of meditation. When the users in our treatment groups experience course content rather than the breathing exercise, their expectations of what meditation is may have been violated and this may have made them less likely to continue. TLDR: When you expect short breathing exercises, a longer guided session might feel like too much.
- Our personalized treatments over-prioritized sleep courses: As we pointed out above, the most common need states selected were anxiety and stress, but our most commonly recommended course in our personalized treatments was a sleep course. Our matching logic may have over-prioritized responses indicating sleep as a need-state relative to stress and anxiety reducing feelings of idiosyncratic fit.
Implications
While we didn’t successfully increase active meditation, we did increase course starts and boosted engagement, learning a lot about what works (and what doesn’t) along the way:
Learning 1: If you want someone to do something, like starting a course, including it in the onboarding flow can drive behavior when users’ motivation is highest.
Learning 2: Adding questions to signal that something is “right for me” can further increase adoption of the key behavior.
Learning 3: It may be hard to outperform the best beginner friendly content — probably because it’s easy to be wrong and harder to be right.
Learning 4: Implementation intentions show promise for helping people make plans when trying to form new habits.
Learning 5: Stronger habit forming interventions like rewarding users with streaks for engaging in a key behavior consistently might be necessary for demanding behaviors like meditation.
Looking to change behavior? Contact us: [email protected]