Crown Levels: A royal redesign

Note: Since publishing this post, Duolingo's homescreen has been redesigned to represent a learning path. With this redesign, we have eliminated Crown Levels. Read more about our new homescreen here!

Recently, Duolingo rolled out one of its biggest redesigns in years: Crown Levels. We wanted to make the learning experience even better by surfacing more challenging content while keeping Duolingo fun and motivating. Six months later, we have some exciting results!

In this post we will share:

• Why it was time for a change
• The research and design decisions that led to Crown Level
• How we designed each level
• How Crown Levels has affected the learning experience

Keeping our mission a priority

Duolingo’s mission is to develop the best education in the world and make it universally available. Unlike many other educational resources, Duolingo is accessible to anyone with a smartphone or computer. This allows us to reach millions of people across a variety of backgrounds. However, being an app means that learners can leave with a single tap. In order for us to have a large impact, Duolingo must be both effective and fun.

How does difficulty impact fun?

We constantly strive to make Duolingo a more effective education product and are currently focused on providing more challenging content for our learners. Difficulty and motivation is currently a hot research topic for learning scientists.^[1] In general, people are not motivated to complete difficult tasks. Surprisingly, researchers have yet to find a point at which a task is so easy that it is boring.

Duolingo has found similar results when we have tried to make our content harder. Learners become less motivated and stop using the platform. With that in mind, we decided to be more ambitious and redesign the app to offer harder content when learners are ready -- without harming motivation.

Redesigning Duolingo for harder content

Duolingo courses are separated into sections called skills. Each skill is responsible for teaching around 30 new words as well as a few grammatical concepts. Skills are centered around a situation (such as travelling) or a specific part of the language (such as greetings or common phrases). Skills are made up of lessons that teach a bite-sized chunk of the words and grammatical concepts in the skill.

Prior to Crown Levels, each skill had a strength associated with it. When you first completed all the lessons in the skill, the skill would be at maximum strength. Over time, the strength would decay, telling you to revisit a skill and review its content. This interface did a great job of showing when to review a skill, but did not give us a place to provide more challenging content in a clear way.

Crown Levels removes strength and replaces it with a new concept: leveling-up. Each skill begins at the introduction level (level 0). When you complete all the lessons in the Introduction Level, the skill is leveled-up and the next row of skills is then unlocked. You can dive deeper into a specific skill by progressing through 4 more review levels that grow in length and difficulty. When you complete a level, you gain a crown, which gets added to the crown count at the top of the screen.

We decided to go with a leveling-up mechanic because we wanted to give learners the ability to see more challenging content without forcing them to go through it if they weren’t interested.

Each level is a step on your learning journey

Each level focuses on teaching words and grammatical concepts in a specific context such as reading, listening, or writing. We picked the context of every level so that they build on each other naturally. Let’s see how each level brings you one step closer to mastery.

Introduction Level: Recognition

This level introduces all the words and grammatical concepts in the skill with two types of exercises: image exercises and assisted recognition exercises. Image exercises introduce words by asking you to pick which image matches the word you are learning. Assisted recognition exercises ask you to translate a sentence from the language you are learning into your native language. New words are highlighted and tapping on them shows a translation. There is also a word bank, which contains all the words in the translated sentence as well as a few distracting words to make the exercise more challenging.

Review Level 1: Assisted Production

This level focuses on preparing you to produce sentences in the language you are learning. This is done through assisted production exercises, where you have to translate sentences into the language you are learning with the help of a word bank. These exercises are much like the assisted recognition exercises of the Introduction Level, but instead of picking words in your native language, you need to pick words in the language you are learning.

Review Level 2: Recall

Up to this point, most of the exercises you see include a word bank. In level 2, you begin translating sentences unaided. This level contains recall exercises, in which you have to translate a sentence in the language you are learning into your native language. These exercises reinforce your ability to read the language you are learning.

Review Level 3: Listening

This level introduces you to listening exercises where you hear a sentence in the language you are learning and are prompted to write it down. These exercises are designed to improve your ability to listen to the language.

Review Level 4: Unassisted Production

The final level focuses on teaching you to produce sentences through unassisted production exercises, in which you translate sentences from the language you’re learning without the help of a word bank. These are the hardest exercises on Duolingo and are essential for learning to write on your own.

Testing Crown Levels

At Duolingo, we test nearly every change to the learning experience with an A/B test. We split learners into two groups: experiment and control. The experiment group is shown the changed experience, while the control group is shown the standard experience. We measure how these two groups use Duolingo to see whether the change improves the experience. A/B tests keep us honest with ourselves. If a test doesn’t perform as we expected, we either try to improve the new experience or move on to something else.

Crown Levels was a large change to the experience, so we first tested it on learners who were new to Duolingo. By limiting the test to new learners, we can test the new experience from the start of their Duolingo journey without worrying about giving them an inconsistent experience unnecessarily. We later tested on all learners and found similar results.

Do learners level up their Skills?

One goal of Crown Levels is to encourage learners to review language concepts in a variety of contexts. However, we do not force learners to level up past the Introduction Level, which means that they could just be completing the Introduction Level and never coming back to the review levels. To measure whether learners were coming back and completing later levels, we calculated the percentage of lessons done at each level over the lifetime of learners. Below is a plot of our results.

From this graph, we can see that on the first day, about 75% of lessons are in the Introduction Level. Unsurprisingly, on their first day learners are primarily completing lessons in new skills. From the third day onwards, less than half of the lessons being completed are in the Introduction Level. Learners are spending more than half their time in the review levels (1, 2, 3, and 4). We can also see that learners continue to spend more time reviewing skills as they continue to use Duolingo.

Has Crown Levels changed how much learners review?

We know that learners exposed to Crown Levels are spending a lot of time reviewing, but we did not know whether they are reviewing more than they would with the old experience. We decided to see how quickly learners were progressing through the tree both in the new and old experiences. Below is a plot comparing how much of the tree the average learner has unlocked over their time using Duolingo.

From this graph, we can see that learners are progressing through the tree about 10% slower than they were before. We checked to see whether this was due to learners completing fewer lessons every day, but it turns out that learners were completing more lessons with Crown Levels! This suggests that Crown Levels causes learners to spend more time reviewing content they have already seen, which is essential for better learning outcomes.

How has Crown Levels changed Duolingo’s content?

With Crown Levels, each level is designed to teach a skill’s language concepts in a different context. However, we were unsure how this would affect the ratio of the types of exercises learners would see. We calculated the proportion of exercises by type in each level, and compared it to the old learning experience.

We found that Review Level 2 lessons have 6 times the recall exercises as a typical old lesson. Review Level 3 lessons contain 3 times the listening exercises as a typical old lesson. Critically, Review Level 4 lessons contain 9 times the unassisted production exercises as a typical old lesson. These are some of the hardest exercises on Duolingo and are essential for learning to write in the language you’re learning.

Is Duolingo still fun?

We know that Crown Levels has led to learners reviewing more content in harder contexts. But is Duolingo still motivating and fun? As mentioned in the beginning, it is essential to our mission for Duolingo to be fun so that we can have an impact on as many people as possible.

We looked at how Crown Levels affected the number of people coming back to the app each day, which showed that it led to a 7% increase in people using Duolingo 7 days after creating their account and a 10% increase in people coming back 30 days later. This makes it one of the most successful A/B tests we have ever run in bringing learners back to Duolingo!

What’s Next?

The Crown Levels redesign was a huge undertaking, and we couldn’t fit everything we wanted. In particular, we believe that there are two major areas for improvement: variety and guidance.

For some skills, the later levels can feel repetitive and long. We are currently working on and testing the following changes:

• Introducing new types of exercises
• Adding the ability to test out of each level
• Changing the number of lessons in later levels
• Teaching the words and grammatical concepts in multiple contexts in later levels

With the previous interface, learners knew that skills needed to be reviewed when they were no longer gold. Now that skills don’t go gold until all levels are completed, we no longer provide that signal to learners. This means that learners can become confused over what they should do next. One of our main focus areas is finding ways to guide learners based on their past sessions and experience.^[2]

We are very excited that Crown Levels has led to learners spending more time reviewing content in harder contexts, and we look forward to continuously improving the learning experience. We hope you enjoy this change as much as we do!