With the recent buzz about Duolingo and education, we thought we’d share our plan for becoming the most awesome language-learning tool in the world. Not only are we free, now and forever, but we take our product and the knowledge you gain über-seriously. So we are busting out the lab coats and nerd glasses to (1) find evidence that Duolingo actually teaches you effectively, and (2) figure out how we can use science to teach you even better. We also wanted to keep you in the loop, so we’ll be blogging about it!
A Customized Language Education
Every time you finish a Duolingo lesson, translation, test, or practice session, you provide valuable data about what you know and what you’re struggling with. Our system uses this info to plan future lessons and select translation tasks specifically for your skills and needs. Similar to how an online store uses your previous purchases to customize your shopping experience, Duolingo uses your learning history to customize your learning experience.
This kind of adaptive and net-native approach to education is pretty unique to us. In contrast, most language-learning software has a rigid, one-size-fits-all curriculum. You can get a personalized education from a small classroom teacher or private tutor, but there are many people in the world who want to learn a language and can’t afford these classes. Moreover, with a traditional “semester” course — be it in a classroom or an online MOOC — teachers can only assess what did or didn’t work once every few months; then they have to wait until the next semester to improve. At Duolingo, our language experts and engineers tweak the site literally every day to make it better for you and a million other language-learners.
Measuring What You Learn
In this post, we want to tell you how we can measure what you learn, and how we determine if our daily tweaks actually improve learning. Most of the education world measures learning with standardized tests, and there is already some evidence that Duolingo does a very good job at improving scores for the WebCAPE university placement test for Spanish. We don’t want to have you take standardized tests all the time, though. That would be slow and expensive to do for every little tweak!
It turns out that the skill points you earn from lessons, tests, and translations — combined with how often you have to “peek” at word meanings during practice sessions — provide a good prediction of what your WebCAPE score would actually be:
For the scientifically inclined, these predictions are only off by 66 points on average, and the correlation coefficient between the predictions and real scores is 0.68. For comparison, people taking the Spanish WebCAPE twice in a row yields a coefficient of 0.86, according to the official WebCAPE website. All of these results are statistically significant (p < 0.00001).
How This Helps Us Improve
When we design a new Duolingo feature to help you learn better, we first test it out with a small fraction of our users to see if these metrics (higher skill points and fewer word peeks) improve. If they do, then we can be confident that the new feature will help you score better on standardized tests, too. So if you are motivated to earn those coins, you are well on your way to mastering a new language. How cool is that?