Sharath Kowligi

There is a lot of evidence showing that the 2 main things to test on the app stores are:

1. Icon
2. Feature graphic (video thumbnail).

The way to approach testing is to be systematic: it's to think about the testing cadence instead of trying to find THE big idea. Because once a team gets into the rhythm of testing, that's when you get better at it.

Once you run the experiments 1/2/3 times and try on iOS what works on Android, you'll know if this is something you can do. If a killer creative on Android doesn't make an impact on iOS, then you can't carry results from Android to iOS.

For icons, headers, copy, etc. testing, use Returned Users (vs. Installed Users).

For most B2C, you have to give the test at least 7 days. It allows you to run about 50 tests a year for one app.

If you want to do more tests you can also split between markets that you know behave the same way for YOU (e.g. could be US/UK) to run tests in parallel.

If you’re getting low CPIs but poor D0/D1 retention, consider integrating the game mechanics shown in the creative into the game. Frame it as having marketing research showing that people are responding to this visual or mechanic.

The first 10 minutes and Day 0 are highly actionable and cost-effective to test. You can use benchmarks from intelligence tools like Sensor Tower to understand how well you’re doing.

To get Day 7 and onwards retention right, you need to get the core needs of the user right, which means all 3 of those things have to work:

• Marketing: the ad
• Marketing design: the story
• Game design: the game mechanic

Achieving internal consensus on these is challenging yet essential for market acceptance.

Session one and session two problems are different:

• Session one: technical obstacles like software not loading or the game being unintelligible
• Session two: users failed to find value in the product and did not return.

To address session two, make sure the promises made in ads and the game’s features align with users’ expectations.

For conversion testing and onboarding testing, use Google Play's 7-day look back window.

At least once a week, report on experiments whether they are done on the paid advertising side, the product side or growth in general and even on the CRM side.

Putting testing cadence first is a great way to build a creative team. It lowers the ego threshold of being right and being wrong and puts the focus on doing things that are meaningful

Ad creatives definitely impact your D1-D7 retention, even on the same channel and with the same targeting, because you will get different traffic. Example: Sharath mentioned an ad leaning more towards “boring gameplay” that showed a 10-20% lift in D1 retention.