An Adotas Q&A with Justin Bauer, Head of Product at Amplitude, examines how in depth understanding of your app users’ group can transform your marketing efforts.
A: Generally speaking, the term ‘cohort’ typically refers to an acquisition cohort: a group of users who all started using your product in the same time period (whether that’s a day, week, or month).
When people talk about ‘cohort analysis’, they usually mean measuring the retention of each acquisition cohort over time, which is often visualized like this:
Q: Okay, so what are behavioral cohorts?
A: Knowing when a user started is important, but the actions they take in your app or website give you a much deeper understanding of your users and how they interact with your product. That’s where behavioral cohorts come in.
A behavioral cohort is a group of users that is defined based on actions they take in your product. Say you have a music streaming app. You’re interested in learning more about users who mark at least 3 songs as a ‘favorite’ on their first day in the app. Here’s how you would define that behavioral cohort in Amplitude:
Q: Nice, that was easy. Now what?
A: Once you have a behavioral cohort, you can apply it throughout Amplitude to learn more about how that behavior affects things like retention, conversion, and revenue.
Let’s walk through a couple examples of the types of insights you can get from behavioral cohorting. In the graph below, we’re comparing weekly retention of users who favorited at least 3 songs, with users who did not. As you can see, the cohort of users who favorite at least 3 songs have higher retention, indicating that the action of favoriting songs may be an important hook in your app.
In this next chart, we’ve applied that behavioral cohort to a funnel so that we can see how favoriting songs might impact the conversion rate from signing up to purchasing a subscription. As you can see, users who favorite at least 3 songs have a much higher conversion rate to a paid subscription: 18%, compared to only 8.8% of users who favorite less than 3 songs.
Based on these results, it seems like it’s a good idea to encourage your new users to favorite more songs on their first day. That behavior is positively associated with both retention and conversion to paid subscriptions!
Q: How can you tell if your analytics platform really has behavioral cohorts?
A: Some companies claim to have ‘behavioral cohorts’, but if you can’t save the group of users and apply it to other charts, like we did with Retention and Funnels above, it isn’t really a behavioral cohort.
For many analytics platforms (including Amplitude), you can specify a Start Action and a Returning Action for a retention report. This provides you with a more detailed look at retention — in the example below, we want to see what proportion of users Sign Up and then later Favorite a Song.
Some companies will call this “behavioral cohorting,” but it’s really event-based retention. You can’t use the above filters to save a group of users and perform further analysis in other parts of the platform. You also can’t specify a timeframe (like within 1 day of 1st use) or how many times a user has done an action.
Remember, real behavioral cohorting will allow you to:
• define a group of users based on actions they take (or don’t take) in your app
• save that group of users and perform further analysis in charts like Funnels, Retention, and Revenue
• understand your user behavior and make better decisions about your product
Your Step-by-Step Guide to Behavioral Cohort Analysis
Now that you know what behavioral cohorting is, it’s time to use it to understand how different behaviors impact your product’s retention, revenue, and growth! Use the following worksheet as a guide to defining and analyzing your behavioral cohorts.
Phase 1: Define your behavioral cohort
What are you trying to learn from your behavioral cohort analysis? Ask yourself these questions to frame your problem and guide your cohort definition.
1. What action or behavior would you like to investigate?
ex. Adding a friend, sharing content
2. What time frame are you interested in?
ex. Within 1st day of use, during a specific date range, or in the last 30 days
3. Is the number of times the user does the action important?
ex. Add at least 7 friends, complete exactly 1 order
4. Are there any user properties that you need to specify?
ex. Only users in the United States, on iOS
5. Are there additional actions that you’d like to investigate in combination?
You can use behavioral cohorting to investigate multiple behaviors, for example, users who added 7 friends AND shared 3 links.
Phase 2: Analyze the impact of the behavior
Once you save your behavioral cohort, it’s time to understand the impact that behavior has on your core metrics. For this worksheet, we’ll use the example of ‘favoriting at least 3 songs.’
Core Metric: Retention Does favoring at least 3 songs impact new user retention? Directions: Plot the retention curve of the cohort who favorited at least 3 songs, compared with the cohort who favorited less than 3 songs. Is there a difference in retention 1 week later? 1 month later?
Core Metric: Conversion Rate Does favoring at least 3 songs affect your conversion rate? Directions: Set up a funnel of the key steps a user takes to ‘convert’ in your product, whether that’s completing an onboarding flow or making a purchase. Look for differences in conversion rate between your 2 cohorts at each step of the funnel — you might find that there’s a key step at which most of your users are dropping off.
Core Metric: Stickiness Do users who favorite 3 songs use your product more often? Directions: To measure stickiness, look at the percentage of users who access your app n days out of a week (ex. 3 days a week). Compare stickiness between your 2 cohorts: this will tell you whether users who favorite 3 songs use your app more frequently than users who don’t.
Core Metric: Revenue Do users who favorite 3 songs spend more? Directions: If you have in-app purchases or e-commerce orders in your app, compare whether users who favorite 3 songs spend more. Some common metrics to look at are LTV, ARPU, and ARPPU.
Core Metric: Key User Actions Do users who favorite 3 songs perform key actions more than other users? Directions: Investigate how favoriting 3 songs relates to how often users do other important actions in your app. In the music app example, you might look at how many times users create and share playlists, or how many times they purchase songs.
About Justin Bauer
Justin Bauer is the Head of Product at Amplitude, where he strives to make it easy for companies to make better decisions from their data. He’s a 2x entrepreneur as well as an alumnus of McKinsey, Stanford GSB and Carleton College.