Thomas Hopkins (Head of Performance & Lifecycle Marketing at Masterclass - online learning) talks about incrementality testing, differences between CPA and CPIA, several ways to assess ROAS so you can assess the profitability of your channels and match-market testing.
You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Lift tests are really hard to do because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).
You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Lift tests are really hard to do because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).
You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Lift tests are really hard to do because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).
Notes for this resource are currently being transferred and will be available soon.
If you look only at your CPA, you see it going up but because it is under your $250 target you keep spending. But each new acquired user costs more.
[💎@05:12] You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Platform reported sum of purchases is usually more than the actual purchases, because channels are overlapping.
It says YouTube because that's where Masterclass spends the most on when it comes to Google.
Facebook is great at cross-device attribution so often over attributes.
[💎@08:55] Lift tests are really hard to test because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
On YouTube, there are examples of seeing 50/60% ROAS when it should be 150% ROAS.
Regression Analysis
The dependent variable is the revenue (or projected revenue) based on your LTV. All the other variables are based on your channel spend.
How did you hear about us survey
The line charts shows the ROAS.
[💎@13:33] Use a "How did you hear about us survey" as another way to calculate ROAS per channel.
Based on the response % per channel, take that response value and take the cost of the spend in that channel to figure out what the ROAS is.
Example: Purchasers from Facebook * Perceived LTV for customer / Cost.
Looking at everything together
Doing this kind of analysis will show you some clear trends and allow to identify where you have unknowns.
Match-Market testing
Match-market testing is designed to define what the best mix of channels is. You can find the recommended Google white paper on match-market testing here.
You can look at 4 channels just for FB/Google:
[💎@16:58] For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
[💎@18:03] For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
[💎@20:20] Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).
If you look only at your CPA, you see it going up but because it is under your $250 target you keep spending. But each new acquired user costs more.
[💎@05:12] You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Platform reported sum of purchases is usually more than the actual purchases, because channels are overlapping.
It says YouTube because that's where Masterclass spends the most on when it comes to Google.
Facebook is great at cross-device attribution so often over attributes.
[💎@08:55] Lift tests are really hard to test because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
On YouTube, there are examples of seeing 50/60% ROAS when it should be 150% ROAS.
Regression Analysis
The dependent variable is the revenue (or projected revenue) based on your LTV. All the other variables are based on your channel spend.
How did you hear about us survey
The line charts shows the ROAS.
[💎@13:33] Use a "How did you hear about us survey" as another way to calculate ROAS per channel.
Based on the response % per channel, take that response value and take the cost of the spend in that channel to figure out what the ROAS is.
Example: Purchasers from Facebook * Perceived LTV for customer / Cost.
Looking at everything together
Doing this kind of analysis will show you some clear trends and allow to identify where you have unknowns.
Match-Market testing
Match-market testing is designed to define what the best mix of channels is. You can find the recommended Google white paper on match-market testing here.
You can look at 4 channels just for FB/Google:
[💎@16:58] For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
[💎@18:03] For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
[💎@20:20] Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).
If you look only at your CPA, you see it going up but because it is under your $250 target you keep spending. But each new acquired user costs more.
[💎@05:12] You have to look at total profit or total projected profit to be able to see if it is going up or down. If it is going down, your CPIA (Cost Per Incremental Acquisition) is most likely above your LTV.
Platform reported sum of purchases is usually more than the actual purchases, because channels are overlapping.
It says YouTube because that's where Masterclass spends the most on when it comes to Google.
Facebook is great at cross-device attribution so often over attributes.
[💎@08:55] Lift tests are really hard to test because the hold out group is seeing ads until you start the hold out. But they have already seen your ads! So you need to run the lift test for a long time.
On YouTube, there are examples of seeing 50/60% ROAS when it should be 150% ROAS.
Regression Analysis
The dependent variable is the revenue (or projected revenue) based on your LTV. All the other variables are based on your channel spend.
How did you hear about us survey
The line charts shows the ROAS.
[💎@13:33] Use a "How did you hear about us survey" as another way to calculate ROAS per channel.
Based on the response % per channel, take that response value and take the cost of the spend in that channel to figure out what the ROAS is.
Example: Purchasers from Facebook * Perceived LTV for customer / Cost.
Looking at everything together
Doing this kind of analysis will show you some clear trends and allow to identify where you have unknowns.
Match-Market testing
Match-market testing is designed to define what the best mix of channels is. You can find the recommended Google white paper on match-market testing here.
You can look at 4 channels just for FB/Google:
[💎@16:58] For match-market testing take 2 geos (or group of geos) that track/behave the same. Instead of splitting the audience in 2 groups for an A/B test, use one group of geos as A and the other group of geos as B. This allows you to test multiple channels at the same time in group A and use group B as control.
[💎@18:03] For match-market testing, start by choosing cities that do not represent too much spend so you can get approval (5% to 10% of your sales should be the maximum). Turn off spend then gradually add your channels and increase spend.
[💎@20:20] Your CPIA curve keeps going up more and more and you want to stop spending when it reaches your LTV. Easiest way is to constantly be monitoring your projected profit (LTV - spend).