Zan Gilani (Principal Product Manager at Duolingo) shares how Duolingo has grown in Asia (doubled their DAU), and how they uncovered global experiments from local research.
Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Local customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating.
Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Local customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating.
Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Local customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating.
Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
Notes for this resource are currently being transferred and will be available soon.
[💎@08:20] Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Duolingo realized that there was a lot of room to grow in Asia: small fraction of Duo’s users, lower DAU.
Duo has grown in Asia through several means:
Global experiments from local research.
[💎@09:30] Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
[💎@10:44] Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
The above looks like a small improvement but it is in reality significant for Duo because they experiment quickly and these add up over time.
[💎@13:53] Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Another example was the improvement of the registration rate in Vietnam.
[💎@16:18] Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Why limit customizations based on local insights?
[💎@18:08] Customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
[💎@18:38] 3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating
Regional experiment example:
[💎@21:56] Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Global experiment example:
[💎@23:24] Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
[💎@08:20] Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Duolingo realized that there was a lot of room to grow in Asia: small fraction of Duo’s users, lower DAU.
Duo has grown in Asia through several means:
Global experiments from local research.
[💎@09:30] Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
[💎@10:44] Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
The above looks like a small improvement but it is in reality significant for Duo because they experiment quickly and these add up over time.
[💎@13:53] Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Another example was the improvement of the registration rate in Vietnam.
[💎@16:18] Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Why limit customizations based on local insights?
[💎@18:08] Customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
[💎@18:38] 3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating
Regional experiment example:
[💎@21:56] Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Global experiment example:
[💎@23:24] Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
[💎@08:20] Asia has more than 50% of global internet users, more than 60% of global mobile revenue and more than 50% of global language-learners.
Duolingo realized that there was a lot of room to grow in Asia: small fraction of Duo’s users, lower DAU.
Duo has grown in Asia through several means:
Global experiments from local research.
[💎@09:30] Some problems (even global ones) are only discovered through local research: market research, metrics deep-dives for specific countries, on-the-ground ux research.
Examples of insights uncovered:
- Indonesia has the best new user retention in the world (by quite a margin)
- Japan has second-longest streaks in the world (after Hungary)
[💎@10:44] Most of the time, the insights uncovered go beyond just understanding your users: they are more actionable and can actually lead to product improvements, especially when they relate to the early user experience. Example: >95% of learners in India have their mobile phone UI languages set to English, which led to an added option to learn English from Hindi even when the device is in English.
The above looks like a small improvement but it is in reality significant for Duo because they experiment quickly and these add up over time.
[💎@13:53] Users might not feel “safe” with the notion of a test, especially in Japan. On the ground UX research in Japan revealed that learners did not feel safe taking a placement test, because the word “test” was making them feel nervous. Learners would then start at the beginner level and would churn because it was too easy. Changing the language led to a +128% placement tests and +9% D1 retention.
Another example was the improvement of the registration rate in Vietnam.
[💎@16:18] Local research can also help you uncover a global problem when the demographics (e.g. age range) of a country are different: in this specific country the issue is visible but globally the impact is hard to spot initially. Example: reg rate in Vietnam 15% lower because of an issue preventing <13-year-olds demo from registering.
Why limit customizations based on local insights?
[💎@18:08] Customization (“forking” the app) is to be treated with caution: it adds complexity in a way that slows down development and testing in hard-to-notice ways. If priorities change, it’s easy for these custom developments to become hard to maintain.
[💎@18:38] 3 different scales for development for international growth:
1. Local, for a specific country (sometimes) - usually concerns localization (example: adapted push notification which led to huge increase in clicks), even though sometimes it’s about product changes (example: WeChat sign in in China leading to +16% registers).
2. Regional experiment (sometimes) - can be run broadly, but it doesn’t make sense to run for everyone. Example: removing paywall during registration after noticing good results in India, because it led to drop-off.
3. Global (most of the time). Example: Applied Japan placement test experiment to all learners by changing the copy to something less intimidating
Regional experiment example:
[💎@21:56] Duolingo realized that there is no need to run experiments locally first, then globally (like they used to): it’s more efficient and effective to run globally immediately. Example: adding coach Duo to placements tests based on Vietnam UX research.
Global experiment example:
[💎@23:24] Japanese customers are more careful about giving app permissions (including notifications), and they are also in general more privacy-conscious. How Duolingo leveraged this: the leaderboard feature was really popular in Japan. So to get more notification opt-in Duolingo added a new angle to their notification prompt mentioning “Keep your next promotion safe”. It led to a +2% DAU and +111% notifications enabled globally!
