Dear product managers: not all metrics matter. Here are the ones that do

Quantitative data is all the rage these days; numbers aren’t just useful, they make us look competent. Better yet, they comfort us and give a sense of control. Data has become the opium of the modern tech companies. Want to look good in a meeting? Just open up a chart and say loudly “I’ve got the data!” Do that often enough, and the promotion is yours. That is if you don’t drive your company to the ground.

Since we can collect so much data about virtually everything people do in our software, we cripple ourselves with information that doesn’t matter. Making the wrong decisions has never been easier.

This post is an attempt to cut through the noise and save time for those who use metrics to gauge the health of their product or company. The metrics I list here belong to one of three major categories: user engagement, business, and customer service.

User engagement metrics

1. Number of sessions per user

This metric is a good starting point for user engagement and behavior analysis. It’s also pretty easy to get; there’s nothing simpler than tracking how often your users login or open your app.

Tip: Avoid averages and take medians over means, as they are less sensitive to outliers and give more robust statistics.

2. Session duration for a cohort, over time

For every new cohort of users, research how much time they spend interacting with your product. Again, take medians instead of means.

Tip: Compare your average session duration for churned and retained users for some eye-opening data. (E.g., those who churned within 30 days and those who stayed.)

3. Number of key user actions per session

Start by selecting user actions that matter most (e.g., clicks on the “like” button), then trace them over time and for different cohorts of users.

Tip: Compare the differences in this metric between churned and retained customers. Do a t-test.

Pro tip: Always use cohort analysis and time normalization for user engagement metrics. This will help you expose the differences in user behavior and their evolution over time.

Business metrics

Customer Service