Duolingo is committed to breaking down barriers to education and ensuring that all students have access to fair, reliable English proficiency testing. We asked Naomi Care, an expert in inclusive disability practices in testing at Assessment MicroAnalytics, what she thought of the role digital testing can have in expanding access for all test takers.
Testing companies often set rigid requirements, inadvertently creating barriers for the countless test takers worldwide with conditions that make it difficult to sit for traditional high-stakes exams.
Thankfully, advances in assessment technology mean it is no longer necessary to enforce many of these requirements. A new generation of digital-first assessments, such as the Duolingo English Test, aim to transform the assessment experience by upholding three key principles: each test taker is unique, testing environments must be inclusive, and the opportunities for improvement are endless.
No two test takers are alike
Disability is far more common than most people realize. In America alone, 26 percent of people have a disability— that’s one in four people! And an even broader set of people have conditions or forms of neurodiversity that can make it difficult to take a test.
For example: traditional English proficiency tests are designed to be taken in-person at a test center, and last for around three hours with limited opportunities for breaks. Here, the mode of administration and rigid time requirement create barriers: a test taker who grapples with chronic pain may struggle to travel to and from their test appointment, or to maintain concentration over the course of many hours. So even if they have exceptional English language skills, they may not perform well on the exam. That’s because they aren’t just being tested on their English proficiency, but also on their stamina and ability to travel!
What are we actually testing?
In the testing world, this is known as construct-irrelevant variance: additional skills or abilities that are being assessed that have nothing to do with what you’re being tested on. A good test aims to reduce construct irrelevant variance as much as possible. Digital-first tests can do this by removing barriers, like the requirement to travel to a testing centre in the first place.
Making an exam shorter can also reduce construct-irrelevant variance. The Duolingo English Test does this by leveraging computer adaptive testing (CAT) to allow the exam to choose questions based on a test taker’s past responses and skill level, ensuring that the test is neither too easy nor too difficult. The result is a shorter test that is just as rigorous and robust as a longer test. By using CAT technology, the barrier of sustained concentration is reduced, but the skills being tested remain the same.
This is just one barrier that test takers face during high-stakes exams. But if test developers prioritize accessibility as a marker of a high-quality test, we can continue eliminating these difficulties.
A better test is possible!
When it comes to being inclusive, computer-based tests have some big advantages over traditional paper-based tests. First, they’re designed to be compatible with assistive technology, such as electronic braille readers for test takers with visual impairments. This means that test takers who have access to computer-assistive technology can access tests using the same technology that helps them at work and school.
But inclusive testing is about more than assistive technology— the design of a test can also affect whether it is easy to access. To promote a good test taker experience (or TTX), the Duolingo English Test uses a clean interface, with minimal scrolling and large font; such a design minimises barriers to test takers who struggle with visual stress or who have conditions that affect their working memory capacity. The opportunities for improvement are endless.
Of course, even with excellent design, a test may still be inaccessible for some. That’s why all testing organizations should have a team available to discuss any personalised accommodations that a test taker might need, just like Duolingo does.
New realities
With the new development of digital-first tests, it’s much easier for organizations to develop inclusive assessments. Everyone deserves the chance to demonstrate their language skills in a fair assessment, and no one should be excluded from a test when they have the ability to shine.
You can learn more about Duolingo’s inclusive approach to high-stakes assessment in our new paper on the Duolingo English Test research website: Improving test validity and accessibility with digital-first assessment.