All about CTR Tests

Item not bookmarked
Resource bookmarked
Bookmarking...
⛏️
Guest Miner:
Sylvain Gauchet
Review star
Review star
Review star
Review star
Review star
💎  x
14

Ivan Spizharskiy (Chief Publishing Manager at playducky.com) explains what a CTR test is, how it can help save time when looking for successful (hypercasual) games and how not to miss a hit.

Source:
All about CTR Tests
(no direct link to watch/listen)
(direct link to watch/listen)
Type:
Webinar
Publication date:
June 16, 2021
Added to the Vault on:
June 3, 2021
Invite a guest
These insights were shared through the free Growth Gems newsletter.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
💎 #
1

A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

01:20
💎 #
2

About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

02:20
💎 #
3

No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.

02:45
💎 #
4

The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

05:14
💎 #
5

There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).

10:19
💎 #
6

Other FAN vs. Feed differences: 
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
- FAN rarely goes below 5% CTR and has high CPM

12:16
💎 #
7

The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.

13:40
💎 #
8

When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres

15:52
💎 #
9

In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

18:34
💎 #
10

Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)

19:30
💎 #
11

There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

22:58
💎 #
12

Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

25:51
💎 #
13

Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

26:40
💎 #
14

Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.

28:41
The "gems" from this resource are only available to premium members.
  • Unlock access to gems from over 155 mobile growth resources
  • Define your preferred categories and receive new relevant gems directly in your inbox
  • Discuss key insights (and any other mobile growth topic) in the members-only community.
Upgrade Your Plan
💎 #
1

A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

01:20
💎 #
2

About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

02:20
💎 #
3

No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.

02:45
💎 #
4

The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

05:14
💎 #
5

There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).

10:19
💎 #
6

Other FAN vs. Feed differences: 
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
- FAN rarely goes below 5% CTR and has high CPM

12:16
💎 #
7

The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.

13:40
💎 #
8

When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres

15:52
💎 #
9

In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

18:34
💎 #
10

Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)

19:30
💎 #
11

There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

22:58
💎 #
12

Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

25:51
💎 #
13

Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

26:40
💎 #
14

Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.

28:41
The "gems" from this resource are only available to premium members.

Gems are the key bite-size insights "mined" from a specific mobile marketing resource, like a webinar, a panel or a podcast.
They allow you to save time by grasping the most important information in a couple of minutes, and also each include the timestamp from the source.

Become a member to:
  • Unlock access to gems from over 155 mobile growth resources
  • Define your preferred categories and receive new relevant gems directly in your inbox
  • Discuss key insights (and any other mobile growth topic) in the member-only community.
Request Access
💎 #
1

A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

01:20
💎 #
2

About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

02:20
💎 #
3

No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.

02:45
💎 #
4

The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

05:14
💎 #
5

There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).

10:19
💎 #
6

Other FAN vs. Feed differences: 
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
- FAN rarely goes below 5% CTR and has high CPM

12:16
💎 #
7

The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.

13:40
💎 #
8

When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres

15:52
💎 #
9

In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

18:34
💎 #
10

Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)

19:30
💎 #
11

There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

22:58
💎 #
12

Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

25:51
💎 #
13

Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

26:40
💎 #
14

Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.

28:41
The access to discussions on each resource is only available to premium members.

Growth Gems members discuss resources and their key insights (as well as other mobile growth topics) in the community. It's the perfect way to dig deeper, ask questions and get additional perspectives.
Upgrade to premium to:
  • Unlock access to key insights from over 155 mobile growth resources
  • Define your preferred categories and receive new relevant gems directly in your inbox
  • Discuss key insights (and any other mobile growth topic) in the member-only community.
Upgrade Your Plan
The access to discussions on each resource is only available to premium members.

Growth Gems members discuss resources and their key insights (as well as other mobile growth topics) in the community. It's the perfect way to dig deeper, ask questions and get additional perspectives.
Become a member to:
  • Unlock access to gems from over 155 mobile growth resources
  • Define your preferred categories and receive new relevant gems directly in your inbox
  • Discuss key insights (and any other mobile growth topic) in the member-only community.
Request Access

You need to be logged in the community to be able to see the discussion below.
You can also head over directly to this topic in the community

Notes for this resource are currently being transferred and will be available soon.

What is a CTR test?

A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook). 

[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

Why use CTR?

[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.

CTR correlation data

[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

Why no clear correlation?

CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.

CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).

CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).

In case of high CPIs the Facebook algo would take too long to optimize.

Facebook FAN vs. Facebook Feed

[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec)

[💎@12:16] Other FAN vs. Feed differences: 

  • FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
  • FAN rarely goes below 5% CTR and has high CPM


[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.

As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.

How to not miss a hit

Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).

[💎@15:52] When testing for a hit through a CTR test, avoid:

  • Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
  • Videos with excessive number of objects on the screen, because the eye focus needs to be clear
  • Actions (FPS) and other “hardcore” hypercasual genres

[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):

  • Color schemes (high CPI impact)
  • Camera angles (high CPI impact)
  • Gameplay variations (high CPI impact)
  • Character appearances (medium CPI impact)

Q&A

CTR correlation when there is a high CPI?

Yes.

[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.

Is there a difference for seasonal games?

Not really, but would still advise against creating seasonal games.

How many color schemes are optimal?

[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

How do you know when to stop a CTR test?

[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

Checklist? 

Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.

What analytics can you get just with a video?

[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off. 

Do you pay attention to other metrics like IPM?

IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.

Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.

Testing on other platforms than Facebook?

In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.





The notes from this resource are only available to premium members.
↘ At this point, you know what to do ↙
Upgrade Your Plan

What is a CTR test?

A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook). 

[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

Why use CTR?

[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.

CTR correlation data

[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

Why no clear correlation?

CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.

CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).

CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).

In case of high CPIs the Facebook algo would take too long to optimize.

Facebook FAN vs. Facebook Feed

[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec)

[💎@12:16] Other FAN vs. Feed differences: 

  • FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
  • FAN rarely goes below 5% CTR and has high CPM


[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.

As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.

How to not miss a hit

Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).

[💎@15:52] When testing for a hit through a CTR test, avoid:

  • Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
  • Videos with excessive number of objects on the screen, because the eye focus needs to be clear
  • Actions (FPS) and other “hardcore” hypercasual genres

[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):

  • Color schemes (high CPI impact)
  • Camera angles (high CPI impact)
  • Gameplay variations (high CPI impact)
  • Character appearances (medium CPI impact)

Q&A

CTR correlation when there is a high CPI?

Yes.

[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.

Is there a difference for seasonal games?

Not really, but would still advise against creating seasonal games.

How many color schemes are optimal?

[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

How do you know when to stop a CTR test?

[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

Checklist? 

Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.

What analytics can you get just with a video?

[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off. 

Do you pay attention to other metrics like IPM?

IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.

Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.

Testing on other platforms than Facebook?

In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.





The notes from this resource are only available to premium members.

The detailed notes taken for a resource are an easy way to see the gems in context to get a better understanding. They also include any relevant visuals from the source.
↘ At this point, you know what to do ↙
Request Access

What is a CTR test?

A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook). 

[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.

[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.

Why use CTR?

[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.

CTR correlation data

[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.

Why no clear correlation?

CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.

CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).

CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).

In case of high CPIs the Facebook algo would take too long to optimize.

Facebook FAN vs. Facebook Feed

[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec)

[💎@12:16] Other FAN vs. Feed differences: 

  • FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream. 
  • FAN rarely goes below 5% CTR and has high CPM


[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.

As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.

How to not miss a hit

Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).

[💎@15:52] When testing for a hit through a CTR test, avoid:

  • Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
  • Videos with excessive number of objects on the screen, because the eye focus needs to be clear
  • Actions (FPS) and other “hardcore” hypercasual genres

[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.

[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):

  • Color schemes (high CPI impact)
  • Camera angles (high CPI impact)
  • Gameplay variations (high CPI impact)
  • Character appearances (medium CPI impact)

Q&A

CTR correlation when there is a high CPI?

Yes.

[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).

Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.

Is there a difference for seasonal games?

Not really, but would still advise against creating seasonal games.

How many color schemes are optimal?

[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).

How do you know when to stop a CTR test?

[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.

Checklist? 

Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.

What analytics can you get just with a video?

[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off. 

Do you pay attention to other metrics like IPM?

IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.

Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.

Testing on other platforms than Facebook?

In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.