Product engagement: the most important metric you aren’t tracking for your SaaS business

Posted on 2019/8/2 9:27:43
读书笔记
By 王佳亮

If you have followed us over at Sherlock, you will likely have heard us say many times that product engagement is the lifeblood of any SaaS business. This is because the entire SaaS business model is dependent on people using (and getting value from) your product consistently over time. The entire model is dependent on product engagement.

In short, creating a method for scoring engagement is essential for every SaaS business. Without a proper way to score the engagement of your users and accounts, it’s pretty much impossible to know what the hell is going on with your business.

As a Founder/CEO, how many times have you asked, “How are our accounts doing?” or “How’s engagement this month?” only to be faced with blank stares from your team.
As VP of Product, how many times have you wondered if your recent releases were helping to drive overall engagement, only to realize that you don’t have a good way of determining that?
As a Customer Success Manager, how many times have you wanted an easy way to prioritize your day’s work – an easy way to determine which accounts are thriving and which are struggling – only to give up and waste the day sending a bunch of random “just checking in” emails?
As an AE, how many times have you wished you could easily see which trial accounts are ready for a call and which ones you shouldn’t bother with, only to give up and refresh your Facebook feed a few more times?

Thankfully, all of these issues can be solved with a good product engagement scoring system. By scoring engagement, you can create essential visibility for every part of the organization that relies on it.

Read on to learn the step-by-step process of creating a solid engagement score for your product.

Four steps for creating an engagement score for your SaaS product

Step 1: Define engagement for your product

The first step in creating an engagement score is a strategic one. In this step, you need to think about what engagement specifically means for your product. Why? Because your product is unique.

Active vs engaged users

A quick note about active vs engaged users: they are not the same.Lincoln Murphy, one of my favorite SaaS thought-leaders, said it best:

“Very often, I see people using the terms ‘active’ and ‘engaged’ interchangeably when talking about user engagement. I think this is a mistake. These terms are related … but do not mean the same thing.

“You want ‘engaged’ users … not just active users. A common definition for engagement for many SaaS apps is: number of logins. Seriously. This is a lot more common than I wish it were (I see it ALL THE TIME), but this is how a lot of SaaS providers measure ‘engagement.’ Logging in is a proxy for an active user, but it is certainly not engagement. Someone might login to your product to find the cancel button. Engaged users are those who are logging in and using the features that are driving value for him/her.”

We completely agree with this perspective. We don’t believe in some universal definition of engagement that applies to all products. Each product is unique. Each product has its own nuances which define successful engagement. In short, it’s important to specifically define what it means for someone to be engaged with your product.

“Each product has its own nuances which define successful engagement”

For example, a B2B productivity tool could define engagement as a certain number of `projects created`, `tasks completed`, `team members added`, `comments left`, `files uploaded`, `projects completed`, etc.

A social networking application could define engagement as `connections made`, `content posted`, `posts liked`, `comments made`, etc.

The point is , your product IS unique. Don’t run away from that fact. Embrace it and create an engagement model based on the important (and unique) activities that are important to your product. Start with a list of engagement activities that a user can take in your product. For example:

Logged in
Added photo
Shared photo
Invited friend
Commented on photo
Edited photo
Posted to Facebook
Posted to Twitter
Opened email
Clicked on email

You will be iterating and refining these criteria over time as your product evolves, but once you have the list, you can move on to step 2 – tracking these activities

Step 2: Start tracking these product activities (aka events)

I’m assuming most SaaS teams reading this are already tracking their important product events. But if you’re not, what are you waiting for? Get yourself a Segment account and get it done.

Seriously. No excuses. Make it happen.

I mean it. Why are you still reading this? Go! Get it done. Then move to Step 3.

Step 3: Weigh each “engagement” event

Now that you are tracking your important engagement events, the next step is to weigh each event based on its impact, or its importance, to overall engagement with your product. This is an essential step due to one simple fact:

Not all activity is created equal.

Think about it: the act of inviting a new user to your product is a more engaging act than simply logging in. Writing a long post on a social media site is more engaging than simply liking a post. Creating a project on a task management application is more engaging than simply completing a single task. And so on.

So you need to weigh these activities accordingly. Create a table that looks like the one below. List your engagement events on the left column and weigh each event on the right, like so:

You can choose to have a score range of 1–10, 1–100, 1–1000, whatever you prefer. The point is, you should give each event a point value that is in line with its value to overall engagement. As a general rule, common, higher-frequency events should have lower point values. Less common, lower-frequency events – those used by “power users” – should have higher values.

Then, for each one of your users, you should add a column for the number of times they triggered each event over a period of time (for example, the past seven days):

Next, simply multiply the event weight by the number of events. Your result should look something like this:

The total of all your individual event values will give you a total engagement score for an individual user.

Finally, just run this calculation for each one of your users and you will have the basis for a quantified user engagement score.

But don’t stop there. The next step is to give it all some meaning.

Step 4: Give it context by normalizing raw scores

If you are going to go through the exercise of creating an engagement score, it’s essential that it is something that can be used across your organization. This is a metric that needs to be “operationalized” by your various teams. This means that your score needs to be in a format that is easily understood and consumed by anyone.

Simply telling your marketing team that a user has a score of 458 isn’t very helpful. How are they supposed to know if 458 is a good score or a bad score? Is that high or low?

But telling them that a user has a score of 91 on a scale between 1-100 is something they can easily understand.

This is why you need to apply a normalization formula to your raw engagement scores. In Sherlock, in order to normalize scores, we use a process called Winsorizing and then apply an exponential function to ensure that differences in the raw scores are more effectively represented.

Step 5: Apply the scores to make them actionable

Once you have a normalized set of user scores, you have done the hard work. All that is left is using these scores in ways that have direct business application. Here are a few ways to make this scoring model applicable and actionable:

1. Rank your users

When every one of your users has an engagement score, that gives you the opportunity to do something amazing: actually rank your users based on their engagement. And this opens up so many opportunities. For example, you can:

Discover your power users and find out what makes them great.
Prioritize customer success efforts to drive great support, identify problem users and growth opportunities.
Drive more personalized marketing programs.
Identify great user research targets.

This kind of user ranking can help you understand your users in the context of their actual engagement with your product – incredibly powerful!

2. Score and rank your accounts

With all your users scored for engagement, you can then aggregate those scores at the account level (which is essential for any SaaS business). As you can imagine, being able to rank your accounts will allow you to:

Understand the overall health of your business.
Prioritize sales efforts by knowing which trial/freemium accounts are more likely to convert.
Prioritize CS efforts by focusing on accounts that are ripe for expansion and those needing some extra support.
Identify which features are most important to your best accounts.

3. Calculate overall score for your product

By calculating a score for each one of your users, you can also aggregate those scores to create an engagement score for your product as a whole.

By tracking this average score over time, you can determine whether or not the work you are doing on your product is actually driving engagement. You also might be able to make some board-level decisions about when to (or whether to) make further investments in the business.

4. Compare populations or cohorts

A user engagement metric becomes tremendously helpful when comparing different populations of users. You can compare the engagement of new users vs. older users, users on a free plan vs. those on a paid plan, users with different access-levels, etc.

The opportunities to gain insights by comparing engagement across segments are endless.

5. Correlate with other business metrics

“Product engagement is an essential metric for any software business”

Ultimately, a good user engagement score isn’t simply a valuable product metric – it’s an essential business metric for any software business. Comparing the levels of user engagement to other business metrics — like sales, retention, growth, LTV, etc — is a great way to ultimately predict and forecast business progress based on engagement levels.

And if you don’t think user engagement is tied to the value of your business, think again. In 2016, it cost Twitter over $1B in market cap.

Keep improving

Peter Drucker is famous for saying: “You can’t manage what you don’t measure.”

If product engagement is an absolute key to success for your software business, then you simply must have a way to measure it. You aren’t going to be able to improve it unless you do.

Of course, building this system yourself is NOT an easy task! We built Sherlock to not only help teams easily create these engagement models, but to help them operationalize engagement as well. For example, with our Intercom integration, you can ship user and account scores to your Intercom account, allowing your CS and marketing teams to use that data when segmenting users or triggering messages. We’d love for you to check it out.

Whether you decide to use Sherlock or do it yourself, we hope this post offers a framework and helps your team get a quantitative handle on engagement.