The Changing Mobile Measurement Paradigm
Apple introduced App Tracking Transparency (ATT) as part of its iOS14.5 update in April 2021. As a result, apps are now required to gain consent to track user activity across apps. After this update, it was estimated that return on ad spend (ROAS) for iOS ads on Meta decreased by 40%.
The most notable change was a shift in the mobile measurement paradigm - from unique advertising identifiers to alternative methods with improved privacy.
Nobody can afford to sit by idly and wait, because the same disruption that took place with iOS14 is coming to Android with Privacy Sandbox, as well as governmental plans for deeper regulation.
In fact, the Privacy Sandbox for Android Beta is already underway (starting in February 2023 with a small percentage of Android 13 devices). Not only that, but iOS 17 features two new major privacy updates. As people's dependence on apps has increased with time, so have their privacy expectations. That’s why it's crucial for marketers to understand these alternate methods if they want to preserve ROAS in the future.
Our marketing optimization framework is built from the experience of running hundreds of projects for high-growth businesses. By the end of the document, you should have a clearer understanding of how to approach marketing optimization on iOS. You and will learn:
- How to optimize the data you send back to ad platforms, lowering cost per action by 10-30%
- How a simple misimplementation of a mobile measurement provider led to a company over-estimating its cost per action by 26%
- Tips for configuring SKAdNetwork to fill the 75% loss in user-level data caused by iOS 14
- The importance of Media Mix Modelling and how this core strategic lever can lead to 20%+ ROI improvements through more effective budget allocation
- The power of incrementality testing and its ability to calibrate your other measurement ROAS estimates by 15%
- The method used by Uber to figure out that a 72% reduction in ad spend only caused a 6% drop in conversions
What is Marketing Optimization?
Most global CEOs agree that marketing, when carried out effectively, is a major driver of business growth. Optimizing your marketing setup helps you to make smarter, more informed decisions, and by extension achieve the best possible return on marketing spend.
Marketing optimization can be broken down into three main constituent parts:
- Data collection - ensuring that you are collecting the right first-party and external data for your marketing needs
- Ad-platform optimization - ensuring you are sending the data ad platform’s algorithms need in an automated and efficient way
- Measurement & Analytics - understanding how effective your marketing is at driving business impact and adjusting your actions accordingly
As advertisers mature and fine-tune their use of advertising channels, they learn how to more effectively collect and send data to ad platforms in an automated fashion.
They grow to better understand the incremental value of each marketing touch, which ultimately leads to more revenue and lower acquisition costs.
Mobile Marketing Measurement Optimization
The introduction of iOS 14 has made marketing measurement more difficult as without the unique advertising identifier (IDFA), it is much trickier to effectively tie users who saw ads (data owned by the ad platform) to users who converted on your site (owned by you).
However, we believe the fundamental framework for solving this problem has remained the same - measurement optimization should be broken down by the types of decision that marketers need to make:
Daily Decisions: Ad Platform Optimization
When it comes to making small optimizations, ad platform automation is far more effective than using human teams. For this reason, marketers need to understand and optimize the data that is sent to each ad platform they use. Their main goal should be to give the ad platforms’ algorithms the richest datasets possible so they can automatically optimize toward their set goals.
Challenge 1: Ensuring Your Ad Platform Has the Necessary Data For Effective Optimization
This data transfer is increasingly difficult in certain geographies (such as Europe with its cookie laws) and on certain platforms (such as iOS with Apple’s App Tracking Transparency). These ‘measurement gaps’ need to be filled whilst maintaining the end-user’s privacy.
Recommendation: Optimize data transfer back to ad platforms
There are multiple ways of achieving this. However, the most obvious is to send user-level data wherever possible - and to fill measurement gaps with new privacy-centric technologies. This approach might involve sending more reliable server-side events to ad platforms via an API (for example, Meta’s Conversions API, which has been known to deliver Cost Per Action reductions between 10-30%).
It’s also important to keep up with the latest ad platforms measurement tools, such as Google’s Analytics for Firebase, or on-device measurement. When implementing Google Analytics for Firebase, Vinted (a European marketplace for secondhand clothing) saw 92% more installs and decreased the cost-per-engagement by 22%.
There are a lot of different approaches to take here - and marketers’ strategy will vary by ad mix - so understanding and implementing those that are relevant to your channel mix will be essential.
Weekly decisions: Attribution and Marketing Analytics
The Importance of a Cross-Channel Source of Truth
When making go/no-go decisions on campaigns, it is important to evaluate these in light of other campaigns currently running.
It’s common for networks to ‘mark their own homework’ and will report on 100% of conversions that they believe they have influenced. As soon as ads are running on multiple channels, marketers will start to see the sum of all conversions across ad platforms being greater than what is found in first-party data:
Marketers need a system that says whether that conversion should be attributed to each marketing touch. In the example above, was it the Meta click that led to the purchase, the Google click, or 50% each?
Attribution on Mobile
Most mobile providers leverage a Mobile Measurement Provider (MMPs), such as Appsflyer, Branch, or Adjust, for these attribution models. For Android campaigns, MMPs will capture user-level signals where possible and attribute credit to the appropriate campaign. For iOS campaigns, user-level data is scarce, with App Tracking Transparency removing user-level signals for those that don’t consent and global consent levels averaging 23%. To fill this 77% user-level data gap, advertisers have to rely on SKAdNetwork as the ‘source of truth’ for campaigns that drive installs. MMPs will use SKAdNetwork data as well as consenting data and other first-party signals in their attribution models.
Challenge 2: Building the Correct Data Foundations for Effective Attribution
Many of the tools and approaches mentioned above revolve around the capture of first-party usage data, including click_ids from deep links and in-app events. To get things in motion, you need to set up first-party tracking.
Recommendation: Make Sure You Have a Reliable Stream of Analytics Events and Have Set Up Your MMP Correctly.
When it comes to mobile attribution, the most common mistakes we notice arise from:
- Companies not setting up their MMP correctly
- Companies failing to reliably capture the relevant analytics events
When auditing a European food delivery firm’s MMP, we identified a flaw that caused an overestimation of CPA for search campaigns by 26%. Fixing this issue helped the client to allocate their marketing spend more effectively.
Different Levels of Data Granularity on iOS
With App Tracking Transparency consent on iOS you receive both user-level data for those who give you consent to track them and anonymous SKAdNetwork data for all users.
The drawback is that consenting users appear in both the user-level data and the aggregated SkAdNetwork data - which means you need to de-duplicate these data sets. The challenge is to count a given user once because that same user can be attributed by two methods.
MMPs have slightly different solutions for de-duplication, however. Appsflyer and Branch allow marketers to achieve a ‘Single Source of Truth,’ where some of your SKAdNetwork buckets will identify users who have consented (i.e. where you have reliable user-level attribution data). You can then remove this from the SKAdNetwork data so you have a single, de-duplicated view.
Adjust, on the other hand, allows marketers to compare user-level and SKAdNetwork datasets side-by-side. Ultimately, both approaches rely on assumptions and each has its own strengths and weaknesses. What is best for you will depend on your ad mix and business.
Recommendation: Optimize Your SKAdNetwork Setup For More Granular Insights
SKAdNetwork is Apple’s privacy-preserving measurement solution. It serves as the primary measurement approach for iOS. On a very basic level, the latest SKAdNetwork works as follows:
This whole system is anonymous. The granularity of the data received in the ‘postback’ will vary depending on how many daily installs the campaign has driven.
If you meet acceptable privacy thresholds, Postback 1 can contain 64 buckets (‘conversion values’) based on the actions users take in the first two days post-install. For example, in cases where the conversion value = 1, some users open the app and take no further action.
In other situations, say where conversion value = 35, users might carry out their first purchase with a value of $5-10. If you fall short of the necessary threshold, you will receive a rudimentary value estimation (low, medium, or high value).
Necessary Actions:
- Combine campaigns to meet the privacy thresholds wherever possible
- Map out signals (within the first two days of download) that predict lifetime value. Marketers can achieve a lot with 64 buckets.
Recommendation: Invest in Marketing Analytics
Regardless of how skillfully you set up your MMP and SKAdNetwork, the reporting features are often insufficient to address crucial business questions that arise. At some point, relying on slow-loading spreadsheets and a single mobile measurement provider will no longer be adequate. As your spending increases, the cost of inaccurate reporting increases with it.
Shifting this data into the data warehouse - and investing in a marketing data mart - will help you get prepared for ever more advanced ad-platform automation and reporting. This might include automated bidding, and analysis, which saves time, introduces a much greater variety of signals, and allows for bid differentiation with a high degree of precision.
Monthly / Quarterly Decisions: Modeling and Experimentation
SKAdNetwork’s attribution model is last-touch, which means that it gives credit to the final click prior to the app install. However, attribution doesn’t fully establish the incrementality of each of your marketing campaigns in making users convert.
Larger app marketers study their results in MMP or marketing analytics and calibrate using additional experiments and modeling. This is becoming increasingly important in light of the privacy changes sweeping the industry.
Modeling
When it’s time to decide your optimal allocation of spend across channels, modeling is critical. Media Mix Modeling (MMM) applies statistical models - mostly around factors like geography and channel-specific spending - to estimate spend efficiency and outcomes, such as revenue. They also help to measure efficiency channels where user-level data is scarce, such as iOS, or non-digital channels like out-of-home.
MMM are complicated to implement. Nonetheless, it has been shown that strategic optimization (including MMM) can lead to a ROAS improvement of 20%+ through more carefully targeted budget allocation. Furthermore, MMM’s predictive nature allows you to simulate performance at different budget levels which helps marketers understand how increasing (or decreasing) budget will lead to changes in marketing performance.
Experiments
Marketers can run experiments to better understand the incrementality of a specific campaign. These experiments typically involve segmenting audiences into test and control groups - and withholding the media in question from one of the audience groups.
Testing the difference between these two groups will enable marketers to better understand the incrementality of each channel. iOS is tricker to test, as user-level data is scarce and often relies heavily on Geo-Testing (showing ads to one geography but not another).
These experiments usually give some insight into the likely incremental cost of acquiring a user through the ad channel in question. For example, Uber ran a geo test to find that dropping Universal App Campaign spend 72% led to a 6% drop in conversions. Armed with this information, the company was able to estimate an incremental cost per user.
Your results can then be used to calibrate both your models and your attribution outcomes, helping both the weekly and monthly decisions. Northeastern University and Meta found that calibration on average corrects MMM-based ROAS estimates by 15%.
Challenge 3: Accessibility and Data Requirements
As marketers move increasingly into the statistical realm, they require more data and greater analytics resources to achieve satisfactorily reliable outcomes. Most of these techniques also call for a data science team and reasonable levels of automation to implement and interpret.
When user-level data was ever present, ad platform tools like Meta’s Conversion Lift made incrementality testing easy for marketers to run. However, without user-level data, you must resort to alternative experimentation approaches, such as Geo-Tests to get these insights.
Recommendation: Build Up Your Measurement Stack Gradually
As a general approach, we would recommend gradually layering on more complexity as your budget and channel mix increase:
- Start with ad platform optimization
- Then focus on attribution and analytics
- Run some experiments to calibrate your attribution
- Build out a Media Mix Model
- Increase the frequency of experimentation and how frequently you read MMM results
The Northeastern and Meta study provides some useful benchmarks as to how frequently you should be experimenting:
Sophisticated marketers (or those with budgets in excess of $20m), should not only be utilizing all of these measurement approaches but also consider how to increase the frequency of experimentation and MMM results read out.
Traditionally, experiments and MMM provide a respectable degree of strategic direction and calibration albeit at a low-time frequency. Increasing the number of insights will improve the accuracy of your calibration and enable you to make better weekly decisions.
It’s worth considering best-in-class MMM tools (such as Recast), or using holdout groups to ensure that your experiments are ‘always on’.
Summary
App Tracking Transparency and other privacy measures have significantly reduced the amount of user-level data available for mobile marketers to measure the success of the ads they run.
With Android following suit, the operating environment is subject to constant change. Marketers must adapt today and implement a range of new technologies to ensure that they can continue to measure and attribute their ad spending effectively.
