Three years ago, Duolingo set up teams dedicated to growing our learner base in Asia. We had felt this region was being underserved—it accounts for more than half the world’s language-learners but represented a small fraction of learners on Duolingo.
And over these last three years, we've homed in on some principles across marketing, course improvement, localization, and product development that not only helped us double the fraction of Duolingo learners from Asia but have also informed our approach to international growth in general.
In this blog post, I’ll focus on just one such principle for product development: “Global Experiments from Local Research.”
Look local…
Our two main forms of local research— analyzing learner data from specific markets and on-the-ground user research—have generated a number of local insights.
In Japan, for example, we found that some new learners with prior English experience were reluctant to take a placement test to jump to more advanced content. As one learner we interviewed shared: “The word ‘test’ scares me. It makes me feel nervous.” These learners would start the course from the beginning only to churn because it was too easy. We remedied this with an experiment that rephrased “take a placement test” to “check your English level”—which doubled placement tests taken and increased retention as well.
Also in Japan, we were surprised to discover that learners were 50% less likely to opt into receiving reminder notifications—receiving notifications is actually one of the biggest factors affecting whether a learner forms a learning habit or not!
In some cases, local research delivers broad insights that would otherwise be hard to unearth.
In Vietnam, we were surprised that the registration rate of new learners was about 15% lower than the world average. Digging around in the metrics, we found a critical issue affecting a specific group: learners under 13 years old, who go through a special registration flow (for privacy reasons). As a result, these learners had a registration rate of 62% compared with 90% for others. The issue, as it turns out, wasn’t Vietnam-specific, and we were able to make a global fix.
How did it take country-specific research to discover a global problem? The registration rate for under-13-year-olds was as bad in Vietnam as it was globally. However, because these under-13-year-olds made up 2.5x the percentage of new learners in Vietnam versus globally, they skewed Vietnam’s overall registration rates in a more visible way than they did for our global numbers.
And yet…
From these illustrations it may be tempting to conclude that the best way to act on these local insights is to create custom experiences for different regions (especially when there are countless stories of companies masterfully localizing their products).
And in some cases, we’ve done just that, such as by offering WeChat (a popular Chinese social media app) as a registration method in China or making some India-specific changes (read more in our blog post on building for Indian learners).
Customization (internally known as forking the app) is sometimes warranted—when it’s necessary or a big enough improvement—but we approach with it caution. Mainly, because it adds complexity in a way that slows down development and testing in hard-to-notice ways. And if our priorities ever change (which they may), it’s easy for those custom experiences to become difficult to maintain or even to silently break.
...so run global experiments (when possible)
Fortunately, we’ve found that many local insights can be translated into global experiments.
Sometimes, we test an experience in a particular country first, and, if successful, bring it to the rest of the world. In the Japan example above, after seeing strong positive results from changing the placement wording in Japanese, we ran this experiment globally and improved placement completion and retention across the board.
In most cases, however, local insights can be used to create experiences for everyone. Looking at the Vietnam example above, we were able to improve the registration flow for all learners under 13 at once, not just those in Vietnam.
Similarly, in order to increase the number of learners opting into receiving notifications in Japan, we launched several experiments which benefited learners globally, Japan included. For example, by moving the opt-in prompt to a more effective location, by changing the opt-in pitch, and by adding the prompt in new contexts, such as when learners get promoted in the leaderboards, or when they run out of hearts.
We believe the core principles of Duolingo are universal. Learners everywhere benefit when we make the app more fun, easy-to-use, motivating, and effective. Through local research and its nuanced application, we’ve not only been able to grow in Asia, but also improve Duolingo for everyone around the world.
