Karan Tibdewal

Use slides for your experiments: excel files are good for internal use but slides are better because they help get the buy-in from different teams and get them involved.

How often do you revisit experiments? Once you have an experiment in place and you have maximized/iterated on it, it is good to have quarterly reviews.

To know which ideas to prioritize, measure the Impact score with Impact = Reach x Relevance x Frequency.

How long to run tests for? A week is often enough but it of course depends on volume. Usually the absolute difference between variants should be more than 200-300 to get proper test results (if you are very small). Do not go for tests that are more than 1-1.5 months.

After 2 or 3 variations of a test idea that does not prove a significant positive impact, do not get too attached to your idea and stop iterating: it is not meant to be.

When it comes to goal metrics, you do not want to only optimize for revenue: you want "leading metrics" that feed in to revenue because there are a lot of variables that go into optimizing for revenue.

To get to the point where more testing = more growth being true you need to have a thorough process in place.

You should not have a "win or lose" mentality. Once you have a significant positive impact you want to scale up the test (apply to 100%) then you want to start iterating on it. Double down on high impact tests and keep iterating.

Set up experiments for success by planning ahead of time and using a sample size calculator tool like the one from Optimizely (link here).

Do a reach audit where you assess how many tests you could run with your user base with your current engagement rate. Example: don't just look at how many people will receive that email and assume a 10% conversion rate. Instead, look at your past performance, look at how many users you can reach and use calculator to figure out time to reach statistical significance.

Use in-app messages to push users to experience the app (e.g. do a workout) and giving them a rewarding message before pushing them to the "impulse funnel".

The traditional "linear" AIDA flow (Awareness → Interest → Desire → Action) is no longer valid. A more realistic journey takes into account the fact that users can go through several different stages, in different order: lapsed users, infrequent users or power users.

Associate the sounds to feelings and leverage it. _Example: Duolingo uses the same sound for their notifications as the sound you get when answering correctly in order to trigger a positive reaction. _

Know your customers and what kind of information/assistance they need. Irrespective of the lifecycle stage a user is at, make sure you have both clear expectations and clear data points that show: \- How often the user intends to use the app \- How the user wants to use the app

Asking users for their commitment goal (e.g. usage per day) can be leveraged to then get the most ambitious users back in the app more aggressively through communications. Example: reminders, challenges, leaderboards.

Fostering a stable relationship irrespective of the customer stage or journey is the key.

Karan suggests a way to think across all lifecycle stages, which requires both the right marketing automation tools and data points.

Use the information that you've collected (e.g. during onboarding) to frame how you ask for permissions.

Onboarding and activation funnels are the main areas of improvement for almost all of the apps (even 5+ years apps!).

Try to find daily habits to which you can attach your app in order to increase daily usage. Example: music when working out, music when cooking, etc.

Reward users coming back to the app by offering something special if they re-engage with the app.

Hitting "impulse" users with a paywall (including free trial) might work very well for you, but you might also want to design a slower approach for all the users that drop off (so you can hit them with the paywall later, after they've experienced the core value of the app).

Feedback and reviews is critical, and should not be limited to when the user has lapsed. Users that start interacting with an app have a high motivation and some strong ideas about what's working and what's not, which can be very valuable (and allow you to keep your power users).