How Incrementality Helps Measure the True Impact of Marketing

Item not bookmarked
Resource bookmarked
Bookmarking...
⛏️
Guest Miner:
Sylvain Gauchet
Review star
Review star
Review star
Review star
Review star
💎  x
7

Shamanth Rao (CEO at RocketShip HQ - Mobile Growth Consultancy) discusses with Maor Sadra (CEO at Incrmntal - Incrementality Tool) and dives into how last touch attribution has always been fraught with inaccuracies and how measuring incrementality might just be the best way to understand what the true impact of your digital marketing is.

Source:
How Incrementality Helps Measure the True Impact of Marketing
(no direct link to watch/listen)
(direct link to watch/listen)
Type:
Podcast
Publication date:
August 17, 2020
Added to the Vault on:
August 22, 2020
Invite a guest
These insights were shared through the free Growth Gems newsletter.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
💎 #
1

It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.

08:46
💎 #
2

If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.

12:24
💎 #
3

Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

13:44
💎 #
4

Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.

16:25
💎 #
5

Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

19:45
💎 #
6

Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.

21:48
💎 #
7

Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.

23:56
The gems from this resource are only available to premium members.
💎 #
1

It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.

08:46
💎 #
2

If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.

12:24
💎 #
3

Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

13:44
💎 #
4

Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.

16:25
💎 #
5

Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

19:45
💎 #
6

Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.

21:48
💎 #
7

Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.

23:56
The gems from this resource are only available to premium members.

Gems are the key bite-size insights "mined" from a specific mobile marketing resource, like a webinar, a panel or a podcast.
They allow you to save time by grasping the most important information in a couple of minutes, and also each include the timestamp from the source.

💎 #
1

It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.

08:46
💎 #
2

If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.

12:24
💎 #
3

Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

13:44
💎 #
4

Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.

16:25
💎 #
5

Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

19:45
💎 #
6

Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.

21:48
💎 #
7

Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.

23:56

Notes for this resource are currently being transferred and will be available soon.

Until now, last touch is the most prevalent attribution method, because there wasn't other option on mobile (vs. desktop where you own the domain and you could look at the entire funnel and come up with your own attribution model).

We went from UDID to IDFA/GAID. Big platforms like FB were self-attributing from day 1 so MMP started sending postback after looking who as the last tap.

  • It was a stretch to think like this but it doesn't make sense: there is a marketing funnel (AIDA - Awareness Interest Desire Action) and giving credit to the last touch is not correct.
  • It also created the notion of fraud, where the game began to be to just get the last click


If an ad tech company claims the last touch on an impression it's called fraud, if Facebook does the same thing it is called self-attribution

 

Multi-touch attribution (MTA)

Slack is a unique example where the conversion point (purchase) ends in web, which allows to trust it better: you know where they come from, etc.

But when you're mobile-only you can't really do multi-touch.


[💎 @08:46] It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.


What FB tells an MMP is "this is the last time we saw that user and therefore we claim the install". The MMP defines if that's true or not (e.g. "no, user comes from Unity") but regardless of the MMP attribution, Facebook will still claim it.


[💎 @12:24] If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.


Incrementality

[💎 @13:44] Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

Marketing is not deterministic. It's not causality that we know, it's correlation.

The average marketer works with between 15 to 60 different vendors/networks (then geos, apps, etc.) and it's easy to get lost in the data.


It's often something outside of marketing that allows marketers to realize that you're spending too much and not getting incrementality. Example: bug, budget reason (example: pandemic).


[💎 @16:25] Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.


Big brands have an advantage because that launch a big campaign then afterwards analyze what happened. They are not trying to conclude that it's deterministic.


On determinism: there's a lot of subjectivity that goes into determining LTV per channel/campaign.


How to assess incrementality per channel

[💎 @19:45] Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

It's a fantastic sales tool but you still don't know if the spend itself is adding incremental value.


incrmntal's approach uses cause interference i.e difference in difference (change). They look at change to create "branches in time" (in retrospect) to come up with recommendations.

Many marketers try to do incrementality tests and they often reach correlation but not causality.

[💎 @21:48] Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.


How to think about small changes?

Example: small campaign on TikTok with $100/day. Maor talked about creative changes instead.


Creative changes are sometimes very subtle but mean a lot. This is where last touch point attribution is a great proxy: conversion rate between click and install in the same vendor means a lot.

SKAdNetwork does not have campaign id but Maor is hopeful they'll add it.

[💎 @23:56]  Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.


Incrementality if you're only 1 channel

If you're starting up with $20k/month change on Facebook, you still want to do sporadic checks to understand how much you're getting out of it. But it's not as important as if you're running several platforms.


incrmntal

Software company with incrementality testing tool for app developers and advertisers: http://www.incrmntal.com/


The notes from this resource are only available to premium members.

Until now, last touch is the most prevalent attribution method, because there wasn't other option on mobile (vs. desktop where you own the domain and you could look at the entire funnel and come up with your own attribution model).

We went from UDID to IDFA/GAID. Big platforms like FB were self-attributing from day 1 so MMP started sending postback after looking who as the last tap.

  • It was a stretch to think like this but it doesn't make sense: there is a marketing funnel (AIDA - Awareness Interest Desire Action) and giving credit to the last touch is not correct.
  • It also created the notion of fraud, where the game began to be to just get the last click


If an ad tech company claims the last touch on an impression it's called fraud, if Facebook does the same thing it is called self-attribution

 

Multi-touch attribution (MTA)

Slack is a unique example where the conversion point (purchase) ends in web, which allows to trust it better: you know where they come from, etc.

But when you're mobile-only you can't really do multi-touch.


[💎 @08:46] It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.


What FB tells an MMP is "this is the last time we saw that user and therefore we claim the install". The MMP defines if that's true or not (e.g. "no, user comes from Unity") but regardless of the MMP attribution, Facebook will still claim it.


[💎 @12:24] If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.


Incrementality

[💎 @13:44] Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

Marketing is not deterministic. It's not causality that we know, it's correlation.

The average marketer works with between 15 to 60 different vendors/networks (then geos, apps, etc.) and it's easy to get lost in the data.


It's often something outside of marketing that allows marketers to realize that you're spending too much and not getting incrementality. Example: bug, budget reason (example: pandemic).


[💎 @16:25] Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.


Big brands have an advantage because that launch a big campaign then afterwards analyze what happened. They are not trying to conclude that it's deterministic.


On determinism: there's a lot of subjectivity that goes into determining LTV per channel/campaign.


How to assess incrementality per channel

[💎 @19:45] Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

It's a fantastic sales tool but you still don't know if the spend itself is adding incremental value.


incrmntal's approach uses cause interference i.e difference in difference (change). They look at change to create "branches in time" (in retrospect) to come up with recommendations.

Many marketers try to do incrementality tests and they often reach correlation but not causality.

[💎 @21:48] Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.


How to think about small changes?

Example: small campaign on TikTok with $100/day. Maor talked about creative changes instead.


Creative changes are sometimes very subtle but mean a lot. This is where last touch point attribution is a great proxy: conversion rate between click and install in the same vendor means a lot.

SKAdNetwork does not have campaign id but Maor is hopeful they'll add it.

[💎 @23:56]  Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.


Incrementality if you're only 1 channel

If you're starting up with $20k/month change on Facebook, you still want to do sporadic checks to understand how much you're getting out of it. But it's not as important as if you're running several platforms.


incrmntal

Software company with incrementality testing tool for app developers and advertisers: http://www.incrmntal.com/


The notes from this resource are only available to premium members.

Until now, last touch is the most prevalent attribution method, because there wasn't other option on mobile (vs. desktop where you own the domain and you could look at the entire funnel and come up with your own attribution model).

We went from UDID to IDFA/GAID. Big platforms like FB were self-attributing from day 1 so MMP started sending postback after looking who as the last tap.

  • It was a stretch to think like this but it doesn't make sense: there is a marketing funnel (AIDA - Awareness Interest Desire Action) and giving credit to the last touch is not correct.
  • It also created the notion of fraud, where the game began to be to just get the last click


If an ad tech company claims the last touch on an impression it's called fraud, if Facebook does the same thing it is called self-attribution

 

Multi-touch attribution (MTA)

Slack is a unique example where the conversion point (purchase) ends in web, which allows to trust it better: you know where they come from, etc.

But when you're mobile-only you can't really do multi-touch.


[💎 @08:46] It's not really possible to offer multi-touch attribution in a mobile environment. Self-attributing platforms create too much overlap: FB, Google, Amazon have essentially the same reach and you're only getting aggregate impressions and clicks data from them but not user-level data.


What FB tells an MMP is "this is the last time we saw that user and therefore we claim the install". The MMP defines if that's true or not (e.g. "no, user comes from Unity") but regardless of the MMP attribution, Facebook will still claim it.


[💎 @12:24] If you own the web domain, you do not need anyone to help with attribution since you have all the information. Unlike on mobile where you go through the App Store and rely on a 3rd party. Example: Facebook click to web goes directly to your domain and you can track users with your own parameters.


Incrementality

[💎 @13:44] Marketing is not deterministic: it's not causality that we know, it's correlation. So the fact that Apple eliminates user-level tracking and goes to an aggregate level is good.

Marketing is not deterministic. It's not causality that we know, it's correlation.

The average marketer works with between 15 to 60 different vendors/networks (then geos, apps, etc.) and it's easy to get lost in the data.


It's often something outside of marketing that allows marketers to realize that you're spending too much and not getting incrementality. Example: bug, budget reason (example: pandemic).


[💎 @16:25] Doing incrementality testing once in a while does not really allow you to continuously optimize what you're doing. Maor knows a casual gaming company that once in a year cuts all advertising and reactivates vendor by vendor.


Big brands have an advantage because that launch a big campaign then afterwards analyze what happened. They are not trying to conclude that it's deterministic.


On determinism: there's a lot of subjectivity that goes into determining LTV per channel/campaign.


How to assess incrementality per channel

[💎 @19:45] Facebook and DSPs (especially for retargeting) offer incrementality testing. If you're a platform you can indeed create control/test groups but you still don't know what these users see outside the platform.

It's a fantastic sales tool but you still don't know if the spend itself is adding incremental value.


incrmntal's approach uses cause interference i.e difference in difference (change). They look at change to create "branches in time" (in retrospect) to come up with recommendations.

Many marketers try to do incrementality tests and they often reach correlation but not causality.

[💎 @21:48] Reaching causality is much difficult than correlation and the problem is that many marketers are trying to do it with the notion of real-time and determinism.


How to think about small changes?

Example: small campaign on TikTok with $100/day. Maor talked about creative changes instead.


Creative changes are sometimes very subtle but mean a lot. This is where last touch point attribution is a great proxy: conversion rate between click and install in the same vendor means a lot.

SKAdNetwork does not have campaign id but Maor is hopeful they'll add it.

[💎 @23:56]  Incrementality testing does not aim at replacing attribution but at removing a blind spot. Last touch point attribution (including with SKAdNetwork) is still your best proxy to know if you're completely wasting money or getting something for smaller changes.


Incrementality if you're only 1 channel

If you're starting up with $20k/month change on Facebook, you still want to do sporadic checks to understand how much you're getting out of it. But it's not as important as if you're running several platforms.


incrmntal

Software company with incrementality testing tool for app developers and advertisers: http://www.incrmntal.com/