Welcome to another episode of the Mammoth Growth podcast! In this episode, Data Consultant and former VP Data of FREENOW Dr. Tim Wiegels joins Mammoth Growth EMEA President Stuart Scott and one of our Consultants, Paddy Doran, to discuss the privacy-related changes coming in iOS 17 and their impact on marketers and data professionals.
Tim, Stuart, and Paddy discuss two major changes to the privacy landscape courtesy of Apple: link tracking protection and privacy manifests. With link tracking protection, Apple will start cutting UTM link parameters that may identify users when they open links from mail, messages, or Safari private browsing. Paddy notes that users can already opt in to link tracking protection on iOS 17 beta, and that in the coming months this could affect user tracking and targeting. Paddy then points out that you can think about privacy manifests as an extension of an app’s “nutrition label” in the Apple App Store, where apps need to disclose the data they collect and how they use it. Third-party SDKs will also need to create privacy manifests, increasing transparency and accountability.
The hosts also discuss the challenges and opportunities these changes present, such as the end of fingerprinting / probabilistic attribution and the role of Mobile Measurement Partners in managing privacy and data. Tim then makes a key observation, that as users’ privacy concerns and Apple converge towards heightened privacy protections, “...that we might not need MMPs anymore because if you push something to the Apple App store, they will be your MMP.”
On the topic of bias in marketing attribution, Tim relates a quick story about setting up mobile attribution during his work with FREENOW. Since they thought only a maximum of 30% of their users on iOS had opted into tracking, they considered using these 30% of iOS users and 100% of their Android users to build their mobile marketing attribution heuristics. But they ultimately decided against this approach. They realized that this large sample size contained mostly Android users, but they knew most of their revenue was coming from iOS users. “So really using this large chunk of non-revenue generating users to predict the revenue generating users didn't make sense,” Tim and his team at FREENOW concluded. With this anecdote, Tim reveals the difference between building a marketing attribution model on enough data vs building one with the right data.
Looking forward, Tim, Paddy, and Stuart emphasize the importance of adopting incrementality testing and uplift modeling to understand the impact of marketing efforts and make data-driven decisions. Paddy advises companies to be careful about plugging gaps with user-level data, since this will likely provide only short-term relief and can quickly turn into a game of whack-a-mole. Tim concludes the conversation by asserting that companies can weather the coming privacy apocalypse by focusing on their KPIs instead of fixating on minute fluctuations in user activity.
Transcript
Stuart Scott (00:05):
Alright, welcome everyone to another episode of the Mammoth Growth Podcast. I'm here today with Tim, who was previously VP of Data FREENOW and is now a speaker and a data consultant. And Paddy, who's a senior consultant at Mammoth Growth and is one of our experts on everything related to mobile attribution. And we are here today to talk about some of the privacy related changes that are coming in iOS 17 and the impact that those are going to have on you as a marketer or a data professional. A very quick disclaimer, nothing we say today should be considered legal advice. Obviously it's important to have a good legal basis for any data processing you do, and you should speak to a lawyer in the markets in which you operate to understand that. So without further ado, I think there are two big changes coming in iOS 17 from a marketing privacy perspective. The first one is link tracking protection, and the second one is privacy manifests. So I think we're going to start with a very brief overview of each of those. And so Tim, do you want to lead us off with link tracking protection?
Dr. Tim Wiegels (01:13):
Yeah, for sure. At first. Thanks for having me and nice to meet you in this surroundings, Paddy and Stuart. Yeah, the first thing, I mean in general, we can just say that Apple told us about the privacy apocalypse, the next generation on the last WWDC after iOS 14.5, which was a lot of fun of us already. We want to talk about later. I think there's two more things, and I think the thing that's most relatable or graspable for everyone is this link tracking protection, which pretty much means that, well, I mean all of, if you have a link, there's UTM parameters, there's Google Click IDs, there's anything else, and Apple will start cutting those. Apple will start cutting those. That means if you open a link from, for example, mail or messages or if you open link in Safari private browsing all the link parameters that might identify you, which from my point of view are almost all of them will be cut, which I mean at first it doesn't sound that bad because it's only opening something for mail messages or private browsing, which I think when you use private browsing, you have a certain idea of what you want to browse in.
(02:24):
And maybe these pages don't even do proper user tracking. But I think the bigger problem is really mail messages. And I'm not sure how everyone else does it, but I use all the Apple things on my iPhone. I use a lot of Google tools on my web browser, but honestly, every link I open comes from mail and I don't use messages that often, but mail has a very large market share. I think it's 60% and almost everyone on iOS more than 90% use Safari. We don't know how many use private browsing, but I think when Apple starts this with private browsing, it'll probably extend a bit and at some point we'll be there in all the browsing modes.
Paddy Doran
I've got
TW (03:06):
The, yeah, go.
PD (03:07):
I was going to say I've got the iOS 17 beta on my phone and in settings you can make it all browsing. So it is optional there for people if they want to just go in their settings and
TW (03:17):
Nice. Nice. So it is there already. Nice. Yeah, that's the first thing. And Paddy, do you want to lead on with a privacy manifest?
PD (03:26):
Yeah, so privacy manifests are essentially just an extension of the nutrition label. So those are the things that you see on the app store, which say as an app, we collect this data about you and we use it for these purposes. There's a couple of interesting things that are coming out of the privacy manifests. One is that third-party SDKs, so people like your mobile measurement provider, your ad networks or any kind of third-party code you have in the app, they'll also need to create the equivalent of what you are creating here. And then Apple will compile it all together to say as an app as a whole, you collect you and the third parties that you have in your app, collect this data about your users and use it in this way. Why this is interesting is because often people used to just turn a blind eye and say, we use this SDK, we don't really know what it does under the hood, but that's their problem.
(04:24):
And now they're exposing this to not only the use of the app but also the app developers themselves. So that's kind of the transparency element of that. There's a couple of two other things which I think are more implementation and maybe technically focused, which will have a much bigger impact. One is there's, I think called Required Reasons. So essentially, and I think we'll talk about this a lot later on, but fingerprinting and probabilistic attribution, whatever you want to call it, have been quite big post iOS 14, and the Required Reasons API will say if I'm an app or if I'm a third-party, SDK, I'll have to explicitly say to Apple, “I need this data for one of these reasons.” And the reasons are really small. It's like, for example, to access the active keyboards, your two choices are either I’m an app, which exclusively makes custom keyboards, or I need a custom keyboard for my app for this specific purpose.
(05:32):
And then as they go through the review process, Apple will easily be able to say, actually you are not this. So you can see a world where the amount of data that these people can collect is significantly reduced. And then secondly, which I don't really know how it's going to play out in practice, but they're starting to see when your app makes HTTP requests to tracking domains, so to known tracking domains. And there's talk of how these requests will get blocked if you don't have ATT, or App Tracking Transparency consent. Again tested it on the beta and my user is still sending requests to Google Firebase, to Facebook, to all of these things even without consent. So I don't actually know what that means in practice, but you can see a world where without ATT consent, sorry, it's been Laurie that's just gone past without ATT consent, these requests getting blocked and actually what does that mean for SDKs in the future? So those changes. Yeah.
SS (06:36):
Yeah. And it seems like the big question at the moment is how broad will Apple go with that list of tracking domains? They could choose to block every MMPs domain or they could choose to be very focused on APIs that they think track more than they should. And I don't think any of us really know at this stage how
TW (06:56):
Far it's pretty much being back at the start of GDPR, right? I'm not sure how much you got in contact with it. You are in Europe, but most of your listeners probably in the States. But that was the same. We got all these awesome new regulations and no one really knew how this fits because it wasn't really business related. And the whole privacy manifest goes very much in the direction because you have this, what is it, article six in the GDPR, which is also in German, so Required Reasons and they're doing exactly the same thing. And we also never knew what are Required Reasons, which Required Reasons are fine, and we always wrote something down, but well until now we never really got challenged on that, but Apple is doing that in another way.
PD (07:41):
Yeah, I think what's quite interesting is, so you can see some developers who have looked into the iOS 17 builds and there's a difference in the way that they're approaching the link tracking protection and the way that they're approaching this privacy manifest and tracking domain stuff. So there's specific links of, so with link tracking protection currently in, sorry, not link tracking protection, ITP in Safari currently on-device machine learning, which identifies trackers and does all this stuff. They're taking that approach there. And then with link tracking protection and this kind of tracking domains, it's a fixed file per iOS release and it kind of gets updated as and when new trackers are identified. So at the moment, the list is actually pretty small of what UTM parameters will get stripped, but actually the tracking domain list is really long, includes stuff like Trustpilot and things like that, which I think will be quite baffling if they actually start blocking those requests.
TW (08:47):
Therefore, the tracking parameters, the biggest thing is probably the HubSpot IDs and the Google click IDs and the Facebook click IDs. That would be the most difficult thing for people in the future.
PD (09:02):
I'm interested in the email side of things because probably where I don't know huge amounts, but what's the big impact from a HubSpot ID perspective
TW (09:12):
If you don't know who actually opened your email. So the HubSpot ID is pretty much the part of when you send any kind of marketing emails or CRM emails and put a link in there and the link will tell you exactly this is this person, this person saw the email, this person opened the email, this person clicked the email. So that's pretty much gone,
SS (09:34):
But you should still be able to track clicks through redirect. Most of the click URLs are wrapped in the redirect anyway, which should continue to work then know.
TW (09:46):
And you also see which user ID came into your system via this link, right, because you track this on your first party site. Yeah.
SS (09:54):
So it's tying the two together, that becomes really difficult. You'll see who clicked an email and you'll see who landed on your website, but do you know and what those people who land your website do, but do you know which email click drove which conversion or which order?
TW (10:07):
Yeah, so it's pretty much losing the IDFA, I think it goes all in the same direction. It's all the direction that we have a cut in the middle and this is the way we have to deal with in the future, or we already deal with since the iOS 14.5 apocalypse.
SS (10:23):
Yeah. So do you want to talk a bit about what you did at FREENOW
TW (10:27):
To tackle that? That's good. I think that's also an interesting story because I'm still talking at a lot of conferences and for many people that don't really use apps or that are mainly web based, this is all very new for us, this is the big pain already happened in 2020 and 2021. 2020, when we started knowing about this whole thing, we realized, oh, we now build all this great hyper user targeting tracking, which is also where I come from. I'm originally a bioinformatician data scientist. I started in industry in the early 2010s where was everything was hyper targeting, everything was great, we had all the information. So I built these setups already like five times in different companies and then at FREENOW, we also built this, we were finished in 2020, which was a huge chunk of work. And then, yeah, we got these awesome news, which in the beginning really made us be like German.
(11:22):
There's this saying the ash of Kice, which is pretty much we were really, really afraid because we were like, “oh fuck, everything is gone.” How are we going to deal with our users anymore? Because FREENOW, back then, I mean the thing is every kind of tech startup is always very numbers focused. So you're usually not focusing on brand too much on out of home too much, but you're focusing on, this is where I get my numbers, this is how I steer, and this is also what we did, which when you think about FREENOW, big mobility company is already not that smart because the main thing you want to do is to, well, people need to see something and then they need to open the app and then they need to get into a taxi. So it's a pretty big media break between the digital and analog world, and that means there needs to be a lot of brand, not a lot of foam and so on.
(12:13):
So what we did was pretty early be like, yeah, shit, we build all this. This is great. I mean the structure is still okay, but we really have to deal with this now. And we have to go full on one level up to the campaign level and really start working in a, I don't want to call it marketing mix modeling because that would be too fancy already, but more in the way of we have to work with incrementality, we have to work with experimentation and we have to use this opportunity to really join our information from brand campaigns out of home campaigns with all our digital information and then start building systems, which you can now call marketing mix modeling, which we also did that really look at what's happening before the funnel, what's happening on the marketing side and what are your target KPIs on the other side?
(12:58):
And the other thing we also realized is yes, this sounds very, very bad, but the good thing is also we are all on the same boat. It's not, back then Uber and Bold had better means of targeting people. They had the same problem, exactly the same problem. So our goal was to just start dealing with this in the best possible way and be like, yeah, IDFA is gone. There was also this big discussion about shall we now start using the people that give us their consent, which is like if you're lucky, a third
SS (13:28):
Usually why do you need consent on both ends as well, right? Not just on.
TW (13:31):
Exactly. Exactly. But I mean as long as you don't get the consent for app tracking at all, then there's nothing, right? And we always talk about, shall we use these 30% of iOS users and the hundred percent of Android users to build heuristics and then work on that? And we're like, nah. Because obviously, which is the case, I think in most western countries and where our businesses operate, you might have a large chunk of Android users, but the revenue comes from iOS users. So really using this large chunk of non-revenue generating users to predict the revenue generating users didn't make sense. And what we also said back then already is like, yeah, this is the start. This is the start where we start doing this. We already saw what happened with GDPR and we were already quite sure we can now start building something for really taking care of the Google Ad idea a lot more, which will also then be gone in a few years. Not sure how many years, but yeah, that will happen for sure.
SS (14:29):
Yeah, I'm an Android user here, so it hurts a little bit to hear. There's no revenue from us, but no, I know it's true, but sometimes it hurts a little.
TW (14:40):
Yeah, I mean, it's not saying that Android users don't generate revenue, iOS users don't
SS (14:45):
A
TW (14:45):
Lot more revenue, which is also not saying what each person does, right?
SS (14:49):
Yes. No, and I think what's always a challenge when you move away from user level tracking towards more aggregate methods, whether it's running incrementality tests or geographic tests or whether it's building medium mix models, is ultimately the granularity and timeliness of that data. So we've spent the last decade having almost immediate data at the user level, and now we're in a world where it's at the channel or campaign level and it's often weekly or a test might even take a month or two months to run. How did you tackle those challenges at FREENOW?
TW (15:38):
Not only at FREENOW, when I started working in the industry, my first job was doing TV tracking and uplift modeling, and that goes in a very similar direction. And we already had the issue back then that the marketing managers were all used to, “oh, I need daily data.” But to be honest, even with the hyper targeting and everything else, daily data was never that smart. It made much more sense to really look at something like maybe a week. I always really liked the rolling week window. That was always quite useful because then data will always change a bit. People will actually cancel their subscription, cancel their order, stuff like this. So most of your conversions also change a day later or two days later. So from my point of view, and I think thank God I work with a lot of very good technical and data-driven marketing people, it was always better to have a little bit more big picture view because otherwise you really also significance rise.
(16:34):
Sometimes you look at 40 users that come in over a week and you look at them daily, that doesn't make any sense. So I think we did this education wise a lot before that we were like, yes, please look at the daily data. And if we have very, very, very strong pushes. For example, FREENOW started private hire vehicles by X in I think at the end of 2019, then we looked at data on a minute level. But again, when you think about the whole concept of marketing mix modeling or incrementality, we knew we do something at 10:00 AM in the morning, which will be a massive push. What does happen on the other side? So it's always like it's more it should be causation and correlation and not really just looking at the thing on a minute basis. You have no idea.
SS (17:23):
And you're right, probably a lot of that data that marketers were using that was very granular and very timely wasn't actually statistically significant. So they were making all plus you could make judgments and make decisions. There's probably an element of people were making those decisions too quickly and with too little data. And actually maybe one of the benefits of this will be people are starting to think much more about whether the data is significant or not.
TW (17:45):
Exactly. And I mean the good thing is also when you look at a week, you have the numbers and if you have the question like, Hey, we started PHV and we did a push, is something happening is PHV increasing? Sure, fine. But if you have this kind of use case, it might also be okay to look on a minute basis on an hour basis, but not like, oh, I started a campaign. Let's see what's happening. But what it actually do with the campaign, what's the reason, what's the targeting? That never really helped. And I mean even if the targeting is quite strict, then if you get two people from your target group and 10 from another group then doesn't tell you anything either.
SS (18:22):
Yeah, it makes sense. Should we go back to iOS 17 and talk a bit about which of the changes are you worried about any of the changes? Which ones do you think are going to have the biggest impact on marketers?
TW (18:36):
I'm not really worried anymore because this whole pain of worrying is gone since 2022. I think the most interesting thing is really that the whole fingerprinting and probabilistic attribution will be gone because I think this is difficult for all the marketing measurement partners. So we already asked ourselves and we talked to AppsFlyer and Adjust directly, so how are you going to deal with that? And they were like, well, we talked to Apple, it's all going to be fine, was fine. Let's not talk about how good or bad scan and Scott are. But I think really the fingerprinting being gone is an issue, which is not a problem for me. So I'm not unhappy about this because I already hated fingerprinting 10 years ago because it's always involving IPs and if you are a very mobile driven company, any kind of fingerprinting using an IP is completely useless anyways, or not useless, but I just don't trust it. So to be honest, I'm not sure whether I should say this, but I'm quite happy that stuff like fingerprinting is gone, that because there was never really something you could deal with.
SS (19:40):
Yeah, that's interesting.
PD (19:42):
I think one thing that we saw is that quite a lot of people were using fingerprinting to kind of fill, they were almost doing it lazily. They're either doing it lazily or to fill gaps where SKAdNetwork just wasn't there. So we saw quite a lot of people pushing ads to their website and then having a deep link or a deep link down from their website to install the app and people were kind of using the MMPs fingerprinting methods to track that deterministically. But I think with well ending SKAdNetwork 4 with web to app and that becoming available there, you're not going to be losing those big chunks of Google search ad conversions anymore. So hopefully we'll see fewer people doing that kind of methodology to get that. But then also I think one thing with iOS 17 that is quite an interesting play from Apple is that by not giving access to these richer data points for the MMPs, their fingerprinting accuracy is going to go down, and B, if you wanted to carry on fingerprinting, it's then an active choice from the app developer to do so. You still probably could send IP, OS, device timestamp, server side to these things, but it goes from, I've got apps, let's say I've got apps via installed on my app and they're collecting all of this data anyway to, I'm going to have to send this data to them and make that active choice, which I think is you can no longer turn the blind eye and you can no longer just say, oh, we don't know what's going on here.
TW (21:31):
Yeah, I'm always a little bit going into the conspiracy theory direction here because I mean, when we had the old iOS 14.5 thing starting, we were like, oh, all the data is gone suddenly. Oh, if you do able search ads, you still get all the data. And if you use the SKAdNetwork, you don't get all the data, but you get a lot of the data. From my point of view, it's going more on the direction that we might not need MMPs anymore because if you push something to the app store to the Apple App store, they will be your MMP.
PD (22:08):
I think that's right. From the attribution perspective, I think, and Stuart and I actually talked about this the other day, their kind of connections to ad networks, especially like the long tail and then also just managing SKAdNetwork, especially as it gets more and more complex with multiple post backs, you have to manage app releases, different versionings, all of this kind of stuff. I think the role of the MMP will kind of move slightly from being this probabilistic attribution machine to actually just a tool that helps you implement SKAdNetwork essentially.
SS (22:50):
Yeah, I think one of the things that I think is really important that MMP is you don't want to have to wait until your next app release has been fully adopted to change your conversion value type schema. So yes, you could build a system where the app pings your own servers and downloads the latest conversion value mapping and then updates SKAdNetwork. But that's a lot of infrastructure to maintain unless you're a very large company and for what I guess an MMP is probably a cost effective enough that it's easier to purchase that than it is to build all that infrastructure yourself. And I guess I believe that we should be experimenting with conversion value mappings and updating 'em over time. And also I guess improving our understanding of which events or which early signals are most correlated with long-term like lifetime value. And so if we're doing that data analysis to become smarter about which users we bid for and which events we send to SKAdNetwork, we do need that mechanism to be able to roll those changes out quickly.
TW (24:00):
Yeah, it's almost learning, but I mean it's like Apple got a lot closer to the MMPs, so let's see how it goes. It's still interesting. And I think there's also a lot of stuff going on behind closed doors because as I said, when we talked about them about iOS 14.5 where we didn't know about this cut and scan network, they were like, oh, it's going to be fine. I'm like, oh, and it was fine in the end, right? It's not perfect anymore. But with the conversion value mapping, you still get loads of information, which I didn't expect to be honest.
SS (24:32):
And I think web to app has actually been one of the biggest gaps to date and I think it's been disappointing to some extent how slow the industry has been to adopt SKAN 4 as a result.
PD (24:47):
Fingers crossed it'll all be fixed in the next three months though.
TW (24:50):
That's
PD (24:52):
The timeline.
SS (24:55):
Do you know anything about this bug with SKAdNetwork 4 that's caused Meta to roll back their support?
TW (25:05):
No, to be honest, I just started with it a few weeks ago because my current customers only have Adjust and Adjust just wrote it out for the smaller customers a few weeks ago. Cool. Paddy, do you know?
PD (25:17):
Yeah, it's something to do with the conversion values kept on getting reset. So you're getting a lot of either null or zero conversion values. And the big problem with it is that SKAdNetwork, that version of SKAdNetwork was tied to that iOS release. So they fixed the bug in I think 16.6, but that's one month until it's in general one month of beta until it's in general release. Then two to three months before you get 50 plus percent adoption. And I think this is the problem that they're going to keep on facing with evolutions of SKAdNetwork is you're going to have some people on three, some people on four, there's maybe a bug in one version, which means that it's going to take three months to fix anything before it can actually be done. And going back to our MMP discussion, being able to manage that and how do you ensure backwards compatibility, how would you make sure that when you get all this data that you can actually use it is probably the biggest challenge with SKAdNetwork going forward.
TW (26:29):
So also mps are going to become the consultant and the dealer with all the privacy stuff coming up from our bigger partners.
PD (26:38):
That's why they're going into SKAdNetwork and data clean rooms.
TW (26:42):
Yeah, true.
SS (26:45):
And do we want to talk at all about the changes on Android and Privacy Sandbox?
TW (26:51):
I mean, it's still not there, right? I'm not sure. Did you read this article a few weeks months ago, which really had a big outcry in my network at least because they did this test and they said, oh, it's all great. We realized everything tracking wise and marketing wise works quite well and we tested this with third-party cookies and we were all like, so you tested your anonymized version with third-party cookie, how? what? Because you couldn't track it otherwise? seriously. I was like, yeah, it's very, very intransparent what they're doing. And this article was really like, huh, I think we all didn't get this. There were a lot of very interesting LinkedIn articles. A friend of mine wrote big caption, “Google, what did you actually do there?” And I think we never got an answer.
PD (27:39):
It was like our topics or the grouping people up into these topic audiences only restricted advertising effectiveness by single digit percentages. But I've also seen Criteo, the big publisher, they did some testing where they compared their targeting with topics and they found that there's, and obviously it was written by their data science team, so pinch of salt on both sides, but their targeting was five times more effective at driving actions than this version. So I dunno where it probably isn't single digits, it's probably not five times worse, but there's still no clarity about where in the middle it actually sits.
SS (28:31):
And I guess Google have a big incentive not to rule it out when the performance is that much worse, right, because they are one of the biggest advertising networks,
TW (28:41):
They'll be shooting themselves on the foot a lot.
SS (28:44):
Yes. Okay. So what's the one most important thing that advertisers should be doing now to deal with these changes and the ones we've seen over the last couple of years?
TW (28:57):
From my point of view, it's really you have to follow your use cases, understand what you actually want to change in your business or how your business is behaving. So what's your target, KPI, what I actually doing to change that target KPI, and then, well, I mean in the end it's all incrementality and uplift testing. For me, it's nothing else anymore because this is what will always be working. It will be a bit more painful if you're very big and if you're doing a lot of different campaigns and markets, but I think people have to get accustomed to the idea that this is where the future is going to. It's also something I said a few weeks/months ago on a conference. In the end, everything you learned 20 years ago in university when you were still young, do that again. That's where we are again, this is back and I mean people in our age at least do it in my age bracket. Paddy, you're probably a little bit younger, we learned this 20 years ago and we know how to do this, so don't be afraid to use that again because the hyper targeting is gone and it'll be gone even more soon.
SS (29:59):
Paddy, what do you think?
PD (30:01):
I think, yeah, I was going to echo that completely. Don't continuously try to plug the gaps with user-level data. It's like a real short-term fix and I think it was Eric Seufert as calls it something whack-a-mole and essentially you can keep on doing it, but ultimately you're going to have to invest in these technologies. So it's probably worth spending time now to truly understand them, implement them properly, see what's right for you. And rather than what happened with iOS 14 where it's like everybody had to try and scramble to fix it, if you can get ahead of these changes on Android that are coming where everyone knows where it's going, it's quite easy to see the future. So starting to prepare yourself now by really upskilling and these other different methodologies would be really useful.
SS (30:57):
And I think it's really important to also be systematic about the lift testing you're doing right. It's very easy to run lots of one-off lift tests, run them all with a different methodology, and it ends up becoming, you just end up with holes all over the place because you've tested this and you've tested that, but you haven't tested the third, fourth, and fifth things. And then also making sure you're bringing that data into your regular reporting because there's no point having a bunch of one-off. You don't want to have to hold the incrementality numbers in your head for every campaign every time you're looking at a dashboard. So I think it's really important that you have a system for integrating that incrementality data into your day-to-day reporting so you can be making decisions based not just on the data you get from the ad network, but also your understanding of how that translates into an incremental number.
TW (31:47):
That's actually a very good point. It's not just this, it's also being the marketing tech or data people in the companies. You have to educate the rest of the company and really make them understand why we are doing this and why this is the right way to go forward and how to maybe change the mindset a bit and steer on these numbers and not on the tiny numbers you had before.
SS (32:06):
Yeah, yeah, I agree. Alright, well thank you both. It's been a great conversation and yeah, look forward to speaking to you both soon.
PD (32:14):
Been fun. Thanks guys.
TW (32:15):
Let's do again. See you soon. Bye-Bye. Bye.
