At Duolingo, we make learning a language fun and accessible to learners all over the world. We use gamification and beautiful design to keep our learners engaged, but we're more than just makers of a game — we're serious about learning, too! To understand how effectively our courses teach, we formally tested Duolingo learners and compared them to language students in university courses. What did we find? Duolingo learners performed as well on reading and listening tests as students who took four semesters of university classes — and in less time. In this post, we’re excited to share everything we learned from this new study.
How did we measure learning?
We wanted to see how well our learners compare to U.S. university students, so we started by finding Duolingo learners in the U.S. who were studying Spanish or French on the app. We contacted some of these learners as soon as they completed Unit 5. This is a good place to measure learning because it's the end of the A2 portion of the course, in line with the A2 proficiency level from the CEFR scale — this is where learners start intermediate material! You can see how much material is covered in A2 by checking out the Spanish and French Scope and Sequence documents! Here you can see a part of the Spanish course and the units, which are marked with castles.
Since we really wanted to focus on how well Duolingo teaches, we were careful to study only learners who didn't know the language before they started our course and who weren't using other classes, apps, or programs while using Duolingo. Using these requirements, we chose 225 learners to test in this study.
We compared Duolingo learners to U.S. university students whose language scores were reported in other research from Tschirner (2016) and Rubio and Hacking (2019). Those university students also studied Spanish or French, and they took two tests from the American Council on the Teaching of Foreign Languages (ACTFL): a reading proficiency test and a listening proficiency test. Using the ACTFL tests meant we could be totally objective in how we measured learning, so we had our Duolingo learners take the ACTFL reading and listening tests, too. We then compared Duolingo learners' scores to university students' scores.
Did we make the grade?
As you might expect, university students' test scores improved a little each semester they kept studying the language. So where did the Duolingo learners' scores fall? How many semesters is learning up to Unit 5 comparable to?
The reading and listening proficiency levels of the Duolingo learners were comparable to the proficiency levels of U.S. university students at the end of their fourth semester of Spanish or French courses, as demonstrated in the following figure:
How long did it take Duolingo learners to complete 4 semesters' worth of study?
Each semester in the U.S. is about four months' worth of study, and language classes might average about 60 hours of class time each semester (estimating 4 hours per week for 15 weeks). That means four semesters of university classes takes at least 240 hours to complete, not counting homework and studying!
The median amount of time the Duolingo learners spent studying to reach Unit 5 was about 120 hours, with some variation between Spanish and French. That's only half the number of hours needed for four semesters of university classes! Duolingo learners of Spanish and French got the same learning value, in half as many hours, and they enjoyed the convenience of learning at their own pace, any time, anywhere. Plus, Duolingo learners never had to pull an all-nighter.
High-quality language education for everyone
Our mission at Duolingo is to bring high-quality language teaching to the world, for free, forever, and these new results show just how much our learners are able to get out of our courses: Duolingo learners reach four semesters of language proficiency in half the time as U.S. university students. But we're not finished yet — we also want to deliver this world-class teaching at even more advanced levels, in every one of our 97 courses, across all language skills.
Check out all the details of our research study in our research report and on our teaching efficacy page. As always, happy learning!
References
Rubio, F., & Hacking, J. F. (2019). Proficiency vs. performance: What do the tests show? In P. Winke & S. Gass (Eds.), Foreign language proficiency in higher education (pp. 137-152). Springer International. Retrieved from http://ebookcentral.proquest.com/lib/wvu/detail.action?docID=5622543.
Tschirner, E. (2016). Listening and reading proficiency levels of college students. Foreign Language Annals, 49, 201-223.
