The goal of product analytics is to understand how users engage with your website or app, so you can improve customer acquisition, conversion, and retention. Product analytics tools let you quickly map the full customer journey so you can enhance the user experience while improving key business metrics. And that means, that with the right product analytics setup, companies can understand which top of funnel behaviors lead to more product engagement and higher revenue. The most successful companies all share a single objective for their product analytics strategies: democratize the decision-making process throughout the company so everyone can make better data-driven choices more often, and more quickly.
While Business Intelligence (BI) tools like Tableau are great for producing financial or structured reporting based on the current state of the business, product analytics tools like Amplitude, Heap, and Mixpanel are different. The latter are event-based analytics tools designed to answer a wide range of questions about the complete customer journey. These product analytics tools can tell you whether a sequence of user actions will make it more or less likely that your users will convert.
Product analytics tools offer an alternative to SQL that is both faster and more accessible. Instead of waiting weeks for one of your data scientists to answer your questions about churn or feature adoption, product analytics tools let you get insights in seconds. Yet, while product analytics tools promise to empower a more nimble and data-centric culture, this is only possible with careful planning, a commitment to change management, and an adherence to high standards of data governance.
In this article we’ll share the elements of an effective product analytics strategy, what many companies get wrong about this vital discipline, and what it looks like when teams get it right.
Table of Contents
The careful implementation of any product analytics platform must walk a fine line between centralization of data, and democratization of tools & processes. On the one hand, everyone throughout the company must take care to abide by proper data governance standards. It’s very easy to produce poor results quickly if you don’t have consistent data.
Yet on the other hand, product analytics tools offer every company an incredible opportunity to nurture a culture of curiosity. Imagine being able to answer any question about your customers’ behavior in seconds. Imagine being able to create precise, targeted lifecycle marketing messaging for specific cohorts of users, because you know those users are highly engaged with your product. And in the same way, imagine being able to prevent users from dropping off because you can act on early warning signs with plenty of time to spare. These are just a few benefits of an effective product analytics strategy.
However, results like these are only possible if you understand the balance between data centralization and democratization. And this whole process will be much easier if you follow principles of good change management:
While our current discussion cannot encapsulate every aspect of change management, keep these steps in mind as we walk through the elements of an effective product analytics strategy - especially our focus on good data governance.
As we’ll see, the elements of an effective product analytics strategy build on each other, one at a time.
Strategy Aligns to Business Goals - Before you consider your product analytics strategy, consider your company’s overall goals, and then your specific team goals. How will you generate more revenue, from more customers, more often, at a lower cost? Now, break this down into individual parts. If you’re focused on increasing revenue, which user actions in your product lead to a higher lifetime value (LTV)? How would you test whether these actually increase revenue, and how would you determine if there are other user actions that might be just as important? Now, prioritize each of these questions, and you’ll begin to align your product analytics strategy to your business goals.
As you work your way through this process, a word of caution is in order. Many product analytics solution vendors have adopted a product-led growth strategy, which means they’ve done everything possible to make their tools easy to use. So, while it’s easy to ask all types of questions in these tools, that includes questions that don’t align with your business goals. While exploring your data can be eye-opening for everyone on your team, it’s better to build targeted queries to answer specific questions vs casting a wide net in the hopes of finding something interesting. Be aware of this as you nurture a culture of curiosity, lest your team get bogged down by questions that don’t move the needle.
Data Governance - The best product analytics solution for your business will be useless, unless you 1) collect & transform data so it’s in the right format for all of your downstream destinations and 2) make sure that data is consistent across all your teams. If your data isn’t usable and optimized for your use case, you won’t be able to answer vital questions that roll up to your business goals.
The best way to quickly ship accurate product analytics is to start with good data. Though if your team already has a hard time trusting your data because of errors or inconsistencies, address this first before you invest in a product analytics tool. Just like with any other tool, feeding poorly formatted or inaccurate data to your product analytics solution can exacerbate the problem.
Self-Serve - This is the ultimate form of data democratization. When we asked, “Imagine…,” earlier, we were directing that to everyone in your company. That’s the ideal, though it takes a lot of planning to make it work in real life. If you set up your product analytics solution to be self-serve, you democratize valuable insights. To put it another way, you’re giving the people closest to the problem the resources to solve the problem as quickly as possible. What does this look like?
But once again, outcomes like these are only possible if you set standards to collect high-quality, consistent data across your organization. The last thing you want is for different teams to create inconsistent definitions for the same event, leading to mismatches and worse. In cases like these, teams spend more time talking about which events they should use, instead of talking about what the product analytics data means in the first place.
Auditing and Documentation - Your product analytics strategy will only be successful if it’s aligned with your business goals, based on consistent data, designed to be self-serve from the very beginning… and sustained by regular audits and thorough documentation. Even if you have the first three in place when you implement a new tool, everything will eventually fall apart unless you check on it regularly to make sure it’s working the way you intended. Hiring new employees? Make sure they understand how to use your product analytics system and where to go if they have questions, so the platform can be truly self-serve. Adding a new tool to your tech stack? Make sure data collected by this solution is in the right format for your product analytics tool, whether you have a direct integration or you plan to centralize data in your CDP.
And above all, keep track of queries in your product analytics tool to make sure they’re connected to your business goals. While these tools can make it easier for people closest to the problem to get the answers they need, leadership can be left in the dark unless they follow these results. Performing regular audits, and even analytics on your analytics, is an excellent way to close these loopholes.
All these aspects of an effective product analytics plan represent a more coherent way to collect, manage, and explore your data. But if that’s all they were, it might be easy to think of them as “nice to have” instead of “need to have.” In reality, this approach to product analytics is a more transparent way to do business, which is why change management is so vital to make it happen. And while the tenets of good change management can help you build a motivated, data-centric culture that combines the best parts of data centralization and democratization, there are some pitfalls you need to be aware of along the way.
There is a misconception surrounding session replay and heatmap tools like FullStory, HotJar, and Contentsquare. If you’re learning about product analytics for the first time, it might be easy to confuse the former tools with platforms like Amplitude and Mixpanel. Yet this confusion can actually lead to an enlightened perspective of all three types of tools, and how they should be deployed in concert to understand user behavior in your product.
First, some definitions:
Session Recording tools do exactly what they say. These solutions allow you to record individual users’ sessions in your product or on your website, and playback these sessions at your convenience. With these systems, you can see how individual users navigate each page and where they drop off.
Heatmap solutions display an aggregate view of users’ actions on a single page. These tools can show you where people clicked on different buttons within each page, how many people clicked those elements, how far down people scrolled, and where they hovered their cursor.
Session recording and heatmap tools are great for generating hypotheses about the user journey based on qualitative data. And in a complementary way, you can use a product analytics solution to verify those hypotheses in a quantitative manner. Here’s one possible workflow:
You can also think about session recording and heatmap tools as an intermediate level of magnification into the user journey, between traditional user research and product analytics. Each method gives you a different perspective on the user journey, and they work best when they’re all used together:
Confusing session recording and heatmap tools with product analytics solutions is a common mistake, one that can easily be corrected by understanding how these systems relate to each other. On the other hand, the following slips-ups - while a bit more subtle - can create even greater obstacles to your product analytics strategy.
While all tools in this space rely on event-based analytics to reveal value moments throughout the customer journey, these solutions diverge in terms of how they collect data. Whereas some tools employ autocapture (they collect all data from every user interaction with your product), other tools require you to be more selective about which data you want to collect before implementation. There are advantages and drawbacks to each approach.
Pros of Autocapture
First, you don't need to plan in advance what you want to track. Product analytics tools that capture everything allow you to prioritize what matters after you already have it. And on a deeper level, you don’t need a dev team to launch a product analytics solution with autocapture. That’s because all these tools only require you to add a tiny snippet of code to your product. Once you do that, you’ll be able to analyze all your data, instantly.
For these reasons, product analytics tools with autocapture will usually be a good fit for:
Cons of Autocapture
We’ve worked on product analytics projects with hundreds of different companies, helping them implement tools with and without autocapture. And throughout all these projects, we’ve seen an ironic paradox emerge. Less technical teams are often drawn to autocapture tools because they don’t require a dev team. But that means everyone on your team needs to understand your product analytics goals inside and out. That means everyone needs to understand the right way to build event definitions, how your product analytics tool works, and how to use it. You might not need devs for an autocapture tool, but you’ll always need a water-tight plan that aligns with your business goals and the principles of good data governance.
Also, these tools can’t scale to large numbers of users, precisely because data governance becomes incredibly challenging as you invite more of your team members to experiment with the tool. If you go with a product analytics tool that employs autocapture, you're setting a very low bar to create data, and a very high bar to govern it. In this situation, duplicate events defined in subtly different ways become more and more common, and the whole system tends to break down. Complicating matters even further, autocaptured events are often too granular, which means they require additional aggregation for meaningful analysis.
Autocapture gives teams a low bar for data creation, which can be an advantage on a smaller scale because it allows you to innovate more quickly. Unfortunately, by the same token, that means autocapture can create huge roadblocks as your team grows.
In addition, product analytics tools that use autocapture simply won’t work for certain types of digital products. For example, some sites built on React will use random strings as the IDs for different page elements, and these strings can change over time. And that means the events will break every time you need to rebuild the site.
And on top of this, there is no autocapture tool that offers a simple plug and play solution for mobile apps. In these situations, you would need to adapt the tool’s SDK manually, and figure out identity resolution pre- and post-authentication. If you were curious about autocapture because it meant you wouldn’t need dev support, these considerations would erase that benefit.
Autocapture can be an excellent choice for your product analytics plan if you have limited resources and need insights ASAP. But remember, more data does not necessarily mean more insights, or even better insights. Think of a vast supermarket, with dozens of aisles containing tens of thousands of different items. If you say, "I'm going shopping," how will you be sure you get what you need? You need a list to spend money wisely, save time, and avoid confusion.
Product analytics data is the same way. Capture it all, and it could turn out to be paralyzing and overwhelming. More data in isolation is not useful. You need to carefully consider the needs and goals of each of your teams in your product analytics strategy.
Whether or not to employ a product analytics tool with autocapture is usually a binary choice. Yet there is another decision you must weigh for your product analytics strategy, where the answer isn’t always so cut and dried. Will you track user actions on the client side, or on the server side? Since the nuances of this discussion go beyond the scope of the current article, please refer to this piece for more details.
Should you use autocapture, or would a different product analytics solution be a better fit? And how should you track user events, on the client side or on the server side?
There are a lot of details to consider when you map out your product analytics strategy. And while the preceding two questions challenge many companies, other details can be just as damaging if they’re overlooked.
In 80-85% of the product analytics projects we work on, we’ve seen clients cling to event naming schemas that are far too granular, independent of autocapture. Clearly, if a user clicked a button within your product, you’d want to know which one. But while you might think you’d need more than a single event to distinguish one button click from another, it’s easy to go too far. If you have four different buttons on your homepage that all have the same Call to Action, you don’t need four different events. It’s possible to create one event that includes clicks on all four buttons, and then analyze all those clicks at both aggregate and granular levels if you set up the right properties for that event. Yet if you create four separate events, that will only create confusion.
On a related note, another detail companies often miss is data formatting. While every type of data has an optimal format, we've seen many organizations capture all their data as a string. In these situations, they can’t filter their data the way they want, and they can’t analyze it for meaningful insights. And these outcomes are independent of which product analytics tool they’re working with.
Another key factor to plan for is identity resolution. Many companies don't pay close enough attention to this aspect of product analytics, especially when users are signing in. There are a lot of details to consider in order to link all of a user’s actions together into a consistent sequence of behaviors. Remember, users will often interact with your product across multiple devices. If you don't resolve their identity across all these different touchpoints, you won’t know it’s the same user and you’ll never understand the complete customer journey. Watch out for these pitfalls when you’re planning your product analytics implementation, and you can avoid a lot of trouble later on.
And finally, keep in mind that your product analytics platform doesn’t exist in isolation. If you want insights from across your customers’ journey, you’ll need to set up your product analytics solution to ingest data from just about every other tool in your tech stack. And that means your data needs to be consistent across all your tools. If your product analytics, BI, and customer engagement tools all have different data (or different names for the same events), that can easily lead to errors.
While data governance is critical, cross-functional collaboration is just as important.
This mistake goes beyond matching up your product analytics strategy with the goals of your business. It’s very rare that teams approach product analytics with these three simple questions:
Usually, we see one team in a company acquire a product analytics platform for their own narrow reasons without thinking about the rest of the business. Ideally, every product analytics discussion should focus on the needs of your customers. And since every team in your company serves your customers, every discussion of product analytics should include members from every team. While the first part of this statement might seem blatantly obvious, it’s often more challenging to get buy-in on the second half. But think about it: if you had a tool that could illustrate every aspect of your customers’ journey through your product, why wouldn’t you want everyone in the company to know about it?
This is also why change management, and executive sponsorship in particular, is so important. If you want your product analytics program to deliver lasting results, you need someone at the right level in the organization who can push it forward as a company-wide initiative. This is the best way to ensure that teams don’t work with the tool in isolation. If only one team is using a product analytics tool and no one else knows about it, it’s going to be a lot harder to convince other teams to try it unless you have executive sponsorship.
Often, success in product analytics comes down to ongoing data governance and steady change management. But there’s another area where we see quite a few companies go astray. It’s the entire purpose of product analytics itself.
Many companies see product analytics as an efficient way to collect user actions, whether that’s clicks, views, or anything else. But really, product analytics is a method for analyzing users’ behavior. We hinted at this earlier, when we suggested that more companies should ask, “what is this tool,” as they try to understand their customers. Product analytics platforms can reveal what your users are doing, and tell you why. If the input for your product analytics tool is data created by users as they get value from your product, the output is context. And you can use that context as the foundation for better, data-driven decisions. But only if you start with a plan.
Alignable created a targeted social network for small businesses to share advice, generate customer referrals, and promote each other. More than 5.5 million members use their site, and in order for Alignable to consistently meet their growth goals, they must do everything they can to understand their users’ behavior on the platform. Their team had implemented Looker for aggregate analytics, but drilling down to a specific user’s actions was impractical. Alignable reached out to Mammoth Growth for assistance.
Mammoth Growth recommended the product analytics tool Amplitude, which allowed Alignable to correlate user actions to their core business questions and other key performance indicators. Once they implemented Amplitude, all of Alilgnable’s cross-functional teams could access and manipulate reports without dropping what they were doing to ask an engineer for help, a huge time savings. Product analytics tasks that used to take days are now completed in minutes. Alignable’s teams are empowered to share their ideas for growth with the leadership team, and they’ve got the data to back it up.
Calendly offers an easy-to-use scheduling tool that 10 million people rely on worldwide. Their rapid growth was fueled by quick decisions from siloed teams, who created many different events for their product analytics along the way. For example, at one point there were 35 different events for the 35 different ways a user could share a Calendly booking link. Challenges like this prompted Calendly’s leadership teams to rein in their product analytics strategy and make it more centralized. For guidance with this complex process, they turned to Mammoth Growth.
First, Mammoth Growth performed a detailed audit to determine how Calendly’s analytics data was flowing through their tech stack. This revealed $400,000 in overage fees and spending inefficiencies, which Calendly could immediately incorporate into their future growth projections. From there, Mammoth Growth was able to cut the number of high-priority tracking events by 90% while boosting the volume of high-value analytics tasks that could be completed. These changes had a cascading effect:
Together, Mammoth Growth and Calendly designed an automated data governance process that was built to last. Now Calendly’s product, tech, and marketing teams can explore their product analytics data with confidence, which means more predictable growth for their entire organization.
Course Hero is an EdTech platform that has recruited a network of over 37,000 faculty members to coach millions of students. Add a large network of users to a vast library of learning materials, and you can see the importance of understanding behavioral analytics at a user level. Course Hero had made significant investment in both BI and product analytics tools like Amplitude, but over time they relied more and more on their BI analysts for key performance metrics. This created a bottleneck in their growth, because their BI analysts were forced to deal with every tiny request related to product analytics. When Course Hero realized a poor implementation of Amplitude would prevent them from reversing course on their own, they contacted Mammoth Growth.
Just like at Calendly, siloed teams at Course Hero had been building their own events that were often inconsistent and redundant. Mammoth Growth identified this as one of the main factors preventing Course Hero from getting meaningful product analytics insights at the individual user level. For example, while both Mammoth Growth and Amplitude recommend behavioral analytics taxonomies track no more than 100 events per product, Course Hero’s taxonomy contained 2,700 unique events.
Mammoth Growth consulted with Course Hero to create a new product analytics taxonomy with only 88 total events. With an updated implementation of Amplitude and a simplified product analytics schema, Course Hero’s non-technical team members could get accurate insights in seconds, freeing up their BI analysts for more complex queries. Now everyone on their team can realize the full potential of self-serve product analytics.
Any effective product analytics strategy must balance a wide range of different factors unique to each company:
In order to address companies’ goals for product analytics, Mammoth Growth developed our Analytics Architecture program. The objectives of this 8-10 week project are a complete audit of a company’s tech stack & data infrastructure, and a roadmap for improvements to their product analytics strategy. The Analytics Architecture program allows Mammoth Growth to develop a deeper understanding of a company’s existing product analytics methods and pain points.
When approaching product analytics projects, Mammoth Growth follows these steps within our Analytics Architecture project:
The outcome of this process is a roadmap for the client’s product analytics strategy: how to execute it, what results you’re aiming for, and benchmarks for success. As Mammoth Growth moves through each of these steps with our clients, we take an agile approach to deliver business value as quickly as possible. Mammoth Growth focuses on the quality of each client’s user data and their data governance policies as this enables faster, more accurate analysis. In addition, this makes the client’s preferred product analytics tool more accessible to a wide range of stakeholders on their team.
If you’re new to product analytics and you’re implementing a product analytics tool for the first time, forget about your North Star metric for a moment. You don’t have to begin with a massive plan that involves 15 - 20 different events. Once you’re confident about the quality of your source data, start with one or two events, and enrich them with relevant properties to get a more complete view of your users. Then add one or two more, and continue from there.
In order to get the most value from your investment in product analytics, start with a plan and start small. This clip is from one of our podcast episodes which goes into much more detail on the relationship between data governance and product analytics. View the full podcast episode here.
Any time we have a question in our personal lives - What’s coming up on my calendar? What will the weather be like tomorrow? Who emailed me? - it’s natural to look at our phones. And in the same way, any time you have a question about your customers’ behavior in your product, you should look at your product analytics tool for answers. Yet this only works if you align your product analytics strategy with your business objectives.
For every company, there’s a unique way to perform this balancing act while empowering a much wider cross-section of employees with faster, more accurate insights. It takes discipline and focus to find the right fit for your team. Contact one of our experts today, and let’s talk about your goals for product analytics.
