Ivan Spizharskiy (Chief Publishing Manager at playducky.com) explains what a CTR test is, how it can help save time when looking for successful (hypercasual) games and how not to miss a hit.
A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.
The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
Other FAN vs. Feed differences:
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream.
- FAN rarely goes below 5% CTR and has high CPM
The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.
When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres
In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)
There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.
The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
Other FAN vs. Feed differences:
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream.
- FAN rarely goes below 5% CTR and has high CPM
The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.
When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres
In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)
There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details here.
The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
Other FAN vs. Feed differences:
- FAN is typically shown in apps/games which means you have a more “core” audience of gamers: more chances for a game to get better test results if it’s not mainstream.
- FAN rarely goes below 5% CTR and has high CPM
The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you want to make sure that your game is viable to advertise in the Facebook Feed as well.
When testing for a hit through a CTR test, avoid:
- Complex ideas that can’t be explained in 2-3 seconds (-> ask people to watch the video and tell you what the game is)
- Videos with excessive number of objects on the screen, because the eye focus needs to be clear
- Actions (FPS) and other “hardcore” hypercasual genres
In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
- Color schemes (high CPI impact)
- Camera angles (high CPI impact)
- Gameplay variations (high CPI impact)
- Character appearances (medium CPI impact)
There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
Notes for this resource are currently being transferred and will be available soon.
A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook).
[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.
[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.
CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).
CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).
In case of high CPIs the Facebook algo would take too long to optimize.
[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
[💎@12:16] Other FAN vs. Feed differences:
[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.
As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.
Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).
[💎@15:52] When testing for a hit through a CTR test, avoid:
[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
CTR correlation when there is a high CPI?
Yes.
[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.
Is there a difference for seasonal games?
Not really, but would still advise against creating seasonal games.
How many color schemes are optimal?
[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
How do you know when to stop a CTR test?
[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Checklist?
Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.
What analytics can you get just with a video?
[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
Do you pay attention to other metrics like IPM?
IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.
Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.
Testing on other platforms than Facebook?
In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.
A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook).
[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.
[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.
CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).
CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).
In case of high CPIs the Facebook algo would take too long to optimize.
[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
[💎@12:16] Other FAN vs. Feed differences:
[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.
As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.
Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).
[💎@15:52] When testing for a hit through a CTR test, avoid:
[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
CTR correlation when there is a high CPI?
Yes.
[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.
Is there a difference for seasonal games?
Not really, but would still advise against creating seasonal games.
How many color schemes are optimal?
[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
How do you know when to stop a CTR test?
[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Checklist?
Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.
What analytics can you get just with a video?
[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
Do you pay attention to other metrics like IPM?
IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.
Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.
Testing on other platforms than Facebook?
In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.
A CTR test is making a video of the future game before actually building the game, and testing the performance of that video on a network (Facebook).
[💎@01:20] A CTR test (using a video without building a game) is not that different from ads. Usual parameters: Facebook feed placement, 4:5, 15-20 sec video, gameplay only (no captions), click optimization, no actual build in the store (target any similar game), measure CTR and CPC.
[💎@02:20] About 3 out of 4 CTR tests (videos) fail, which means that it saves you from having to actually build the game.
[💎@02:45] No matter if you start with a CTR test or a CPI test, you still have to go through all the stages of the hypercasual loop to assess if you can have a scalable game. However, going with a CTR pipeline (video first) instead of a CPI pipeline (minimalistic build first) can save you a lot of time. Estimate: 130 days CTR pipeline vs. 240 days. CPI pipeline. Details below.
[💎@05:14] The challenge with CTR tests is that there is a correlation between a successful CTR test and a successful CPI test but it is not always clear because there are a few outliers. For Ducky, a few prototypes had low CPIs and low CTRs.
CTR uses click optimization but CPI uses install optimization and this can result in different FB audiences.
CTR tests are limited to feed placements but when doing CPI you’re looking for the best CPI so you use all placements (and FAN placements are different).
CTR has only 1 conversion stage (1 click) but the CPI has 2 conversion stages (1 click then another one to install).
In case of high CPIs the Facebook algo would take too long to optimize.
[💎@10:19] There are differences between Facebook FAN and Facebook Feed that result in outliers (successful CTR test not correlated with a successful CPI test), and the main one is that in the feed users can skip/scroll through but on FAN there are often skip delays (5 to 15 sec).
[💎@12:16] Other FAN vs. Feed differences:
[💎@13:40] The reason to not use FAN for a CTR test is that FAN usually has a high CTR and costs more. If you use FAN, you’re also running the risk that your game be only placed/promoted on FAN but you you want to make sure that your game is viable to advertise in the Facebook Feed as well.
As you can see below, the attention curve (view-through rate) is very different fro a video placed in the FB feed or on FAN.
Ducky believes they understand how to not miss a hit even with just CTR tests (they’re a publisher so you can test with them).
[💎@15:52] When testing for a hit through a CTR test, avoid:
[💎@18:34] In Ducky’s experience: out of several CTR videos for one game, the one with the highest CTR on a CTR test will have the best CPI on a CPI test. Plus, you are already A/B testing to learn how to best build the game itself later.
[💎@19:30] Here are the best things to test during your CTR tests to learn how to best build the game (see example below):
CTR correlation when there is a high CPI?
Yes.
[💎@22:58] There are some cases where you can have a high CTR on a CTR test and a low CTR on the CPI test. It might be because you have something that looks very eye-catching and appealing as a video but not as a game (i.e. when people realize it’s a game they do not download it).
Low CTR on CTR test but high CTR on CPI test. This can happen almost only if you’re including FAN when running your CTR test.
Is there a difference for seasonal games?
Not really, but would still advise against creating seasonal games.
How many color schemes are optimal?
[💎@25:51] Aim to have 4 videos for your CTR tests. You can try a lot of different color schemes but some are more preferable: blue, purple and green. Also play with brightness and/or contrast (especially on important objects).
How do you know when to stop a CTR test?
[💎@26:40] Stop A/B testing through CTR tests when you’ve done one or two more tests that haven’t brought any improvement. You can still test later, when you’re at the CPI test stage.
Checklist?
Start testing as soon as possible, don’t do extensive polishing because at one point it doesn’t bring you any advantages.
What analytics can you get just with a video?
[💎@28:41] Just with the video of your CTR test you can learn how your audience reacts to your video: you’ll see different video attention curves, which will help you understand which parts are the most interesting and when people drop off.
Do you pay attention to other metrics like IPM?
IPMs might be very high if you’re promoting on FAN (strong “core” audience). The higher the IPM the better.
Looking at IPM can be especially interesting compared to CPI when there are high CPMs, like before Christmas.
Testing on other platforms than Facebook?
In rare cases, Facebook might not be representative of the audience. In these cases they test on Snapchat and TikTok as they are the 2 platforms with the fastest optimization algos.
