Onboarding and Activation
Layer your experiments by starting new tests while waiting for retention data from previous tests to mature. Ensure test hypotheses are independent to maximize testing velocity without compromising data integrity.
"Test A runs for 2 weeks, but once it's paused, it's going to need another 8 days for all the retention data from those trials to come in. While test A is paused, I start test B so that by the time test A's data is mature, test B will have some other data that I'll have with some other variant that's improving or not improving. The key to experiment layering is picking variants where in test B, my hypothesis is not dependent on the results of test one or test A."
The Secret to App Conversion at 6:53 · Featured in Growth Gems #133
Sylvain's take: This is great advice, and why prioritization should not just follow “scoring” (e.g. RICE score) which can tie you down in analysis and rollout sequencing.

Onboarding and Activation
Recommended tests to experiment with app2web:
- Same paywall you have now but send purchase traffic to web checkout
- If > 70% of your users purchase annual plans, test only sending those purchases to web checklut
- If you collect emails, send an offer when users hit the paywall that links to your app2web funnel
The Secret to App Conversion at 21:00 · Featured in Growth Gems #133
Onboarding and Activation
Invest in no-code testing tools and advanced analytics to enable non-technical team members to run growth experiments independently. This removes development bottlenecks and allows testing at the speed of user acquisition rather than engineering sprints.
"Use no-code tools that allow PMs, marketing and growth people to run tests to get around sprint prioritization or apply cadences. They buy or build testing tools and to increase the testing velocity. They improve or buy the right analytics, to your point. Things that allow you to predict and accurately measure the right KPIs to be able to run experiments at scale at the speed of your user volume, not your dev resources."
The Secret to App Conversion at 3:43 · Featured in Growth Gems #133
Sylvain's take: Of course this plays into Superwall’s narrative. Still, tools that enable rapid experimentation and empower the growth team can be very powerful.


Onboarding and Activation
Use a growth lever framework to categorize your tests (pricing, design, messaging, placement, frequency, personalization) and avoid testing across multiple categories simultaneously. This ensures you can isolate what's driving performance improvements.
"At Superwall, we built this framework that we use to basically bundle types of growth tests into different categories. The way I work with customers, is I rarely blend multiple categories at once. Pricing and packaging, payroll design, messaging, placement, frequency, and personalization fit into separate categories, and you rarely ever want to test across these in the same paywall variant."
The Secret to App Conversion at 5:03 · Featured in Growth Gems #133
Sylvain's take: I like this. A good way to diversify your bets when planning your paywall tests.
I’ll add that it’s rare for onboarding experiments NOT to impact paywall conversion and revenue...So you need to take that into account as well.

