At Duolingo, striving for excellence is one of our core operating principles. We strive for excellence in our mobile app by devoting ourselves to a high-quality experience for every learner. As part of our commitment to building for a globally diverse audience, we've dedicated our efforts across the plethora of languages, operating systems, and mobile devices that our learners use. We run hundreds of A/B tests simultaneously, and staying on top of all possible versions of the app a learner might see depending on their user interface (UI) language, course, and device can become overwhelming. Until last year, there was no easy way to visually compare the same learner experience across multiple languages and devices to ensure a smooth, enjoyable experience for everyone.

Enter Bird's Eye.

The birth of Bird's Eye

Like many great ideas at Duolingo, Bird’s Eye was born out of our annual hackathon: a collaborative event where employees across all areas of the company work together to develop brand-new ideas and passion projects.

Bird's Eye is an internal web application at Duolingo that allows us to quickly view and filter app screenshots across dozens of platforms, devices, and language combinations. In partnership with Senior Brand Designer Jack Morgan, I presented a working demo that enabled internal employees to explore app screenshots from the last several versions of the app on our new platform. The project generated a great deal of enthusiasm across our diverse product, engineering, and design organizations, and so I was provided with the opportunity to fully develop Bird's Eye into a viable tool.

A Bird's Eye view

Bird's Eye has a series of dropdown menus that contain sets of filters for rendering screenshots. Duolingo employees can filter based on learner journey (a path followed by a typical Duolingo learner, such as starting a lesson), UI language, app version, device model, and more. Screenshot thumbnails that fit the filter criteria display in a series of carousels on the right side of the screen.

If a Bird’s Eye user wishes to see a screenshot in more detail, they can click on a thumbnail for an enlarged view and continue to scroll through screenshots in a larger view. If a bug is spotted in any screenshot, the user can quickly report it using the Report Bug button. Clicking this button automatically reports the issue and any helpful information about it directly to our internal bugs team for triage.

While the majority of screenshots rendered in Bird's Eye are static screenshots, we currently supplement a number of learner journeys with animated gifs. This allows us to see how the app transitions from one screen to another.

How it works

At Duolingo, we release a new version of our learning app every week for each of our supported platforms (iOS, Android, and Web). As part of our automated release process, we trigger dozens of end-to-end automated UI tests, which navigate through a multitude of scenarios in the app in 23 languages on 11 devices and web browsers. This process ensures that new and existing screens, buttons, and lessons work as expected for our learners. We take hundreds of screenshots in each test run, and these are uploaded to the backend after the test runs are complete.

We upload more than 5,000 screenshots per weekly app release to Bird's Eye. A Python script creates a data manifest of the uploaded images, which specifies categorical information about each screenshot. This categorical data includes the UI language, device, app version, and other useful details about the screenshot. The Bird's Eye JavaScript web application parses the data and renders screenshots based on filtered data queries from the user. We use Tamper to serialize and compress the screenshot data, and we use PourOver to allow the user to make quick filter queries on our large data set (currently at more than 60,000 screenshots).

Impact of Bird's Eye

More and more teams across Duolingo are adopting Bird's Eye into their workflows. After adding Bird's Eye support to our internal analytics tools, we have observed an average of 40 users in Bird's Eye per weekly app release. Here are some of the use cases we're seeing:

• Our localization team has been diligently using Bird's Eye to ensure the accuracy of the translations within our app. They also make sure that our translations fit well on devices of various sizes. Bird's Eye has increased their productivity by 3.5x.
• Our quality assurance specialists have been using Bird's Eye to supplement or replace time previously spent manually testing before app releases. By quickly filtering through screenshots of the same workflows in different UI languages, they have removed redundant manual testing scenarios. Bird's Eye allows them to quickly spot and report visual bugs.
• Product managers and designers have used Bird's Eye screenshots to ensure that new features appear as expected. We've received feedback that Bird's Eye is a convenient place to access screenshots when coming up with new ways to enhance the learner journey in the app.
• Bird's Eye was particularly useful for our New Year's Duolingo Plus subscription promotion, as it allowed the members of our Monetization team to quickly view each screen where the discount offer appeared in the app. They verified that the promotion looked and behaved according to our product specifications on nearly a dozen devices.
• We're rolling out a new feature that will allow our developers to trigger Bird's Eye on GitHub pull requests, allowing them to see how their changes look on multiple devices before merging their code.

If you're interested in building helpful tools like Bird's Eye and making an impact on millions of language learners around the world, Duolingo is hiring!

Special thanks to John Bush (Automation Engineer II), Yijing Chen (Automation Engineer I), Jack Morgan (Senior Brand Designer), and Terrence Wong (Automation Engineer II) for their contributions to Bird's Eye.