Nobody likes waiting, especially when opening an app. We noticed a clear trend: the faster our app started, the more users stuck around. As part of our ongoing efforts to improve Android performance, we added baseline profiles, which reduced startup time by ~30%!
But like all good engineering stories, the road to optimization wasn’t without challenges. We learned a lot while navigating testing setup issues and incompatible third-party libraries. Here's what happened behind the scenes, what we learned, and how it all paid off.
What are baseline profiles?
Android apps store their code as bytecode in dex files, which your device must convert into machine code before the app can run. To understand why baseline profiles matter, let’s first explore how Android’s compilation methods have evolved.
Imagine you're assigned a book for a foreign-language class, but you skipped your Duolingo lessons and haven't learned the language yet.
For apps like Duolingo, which update weekly, cloud profiles and local optimizations are often reset before they can do much good. Baseline profiles let us tell devices which methods and classes matter most so that they are compiled during installation. Now, both our learners and their devices can pick up a new language a whole lot faster.
The journey
On paper, integrating baseline profiles looked straightforward. But spoiler alert: it wasn’t.
While we followed the baseline profile setup instructions successfully, a bug in the macrobenchmark library made testing difficult. Instead of running macrobenchmark tests, which are the recommended testing method, we side-loaded profiles and saw great results! We shipped the changes, checked our metrics, and... nothing. Our metrics stubbornly refused to budge. Cue the sad violin music.
Confused, we reached out to Google’s baseline profile experts and they broke the news: our profiles were entirely invalid. Why? During the build process, the app’s dex files are used to convert the profile into a binary format. However, something was modifying our dex files after this conversion, invalidating our profile entirely.
The culprit: a third-party library. We worked with the library maintainers to help reproduce and debug the issue. After confirming working baseline profiles in our APKs, we celebrated. But hope is a dangerous thing.
As it turns out, APKs aren’t our final product—our app ships using Android App Bundles (AABs). When we tested those builds, we experienced a serious case of déja vu when the dreaded exception popped right back up. After some sleuthing, we identified that a different third-party library was responsible this time. After we filed our bug report, they discovered that two dex files were getting swapped when the APK was packaged, invalidating our profile once again
Unfortunately, this bug only affected niche app setups, making it low-priority for the library’s maintainers. So, we improvised. While waiting for a fix, we added a step to our build process that essentially regenerates the profile using the modified dex files.
With the dex file drama behind us, we were eager to see how baseline profiles improved our app’s launch time. Around this time, the macrobenchmark library was updated, allowing us to finally run proper tests—but we immediately hit another snag. One of the third-party libraries from earlier was now compressing baseline profiles, preventing our benchmark tests from running. Even worse, our experts from Google let us know that compressed profiles add unnecessary CPU overhead at install time and hurt startup performance too.
We tried working around this ourselves, but eventually had to loop in the library maintainers. After some back-and-forth, they acknowledged the issue and promised a fix in their next release. Once their update arrived, our benchmark tests ran smoothly, and we finally saw the long-awaited performance improvements of baseline profiles!
Results
- Our metrics showed a ~30% improvement in startup time, with macrobenchmark tests showing a 25-40% gain.
- Our JIT thread went from being busy 25% of the time to just 3%
Before:
After:
- There were also benefits beyond app startup: reduced jank for music lessons and a path to adopting Jetpack Compose with less concern for performance hits.
Learnings
Testing baseline profiles can be challenging.
Make sure to account for how your testing setup differs from production! Macrobenchmark tests weren’t an option for us initially, so we tested by side-loading profiles. When side-loading, dex files are modified before the profile is converted into its binary format. In the actual build process, it happens after, which is what resulted in an invalid profile.
At Duolingo, we love A/B testing everything. But since baseline profiles are compiled when the app is installed, we can’t run experiments. This makes it difficult to identify how much improvement in our metrics comes from baseline profiles. Luckily, we can now run macrobenchmark tests which helped us validate performance gains!
Get help from experts sooner rather than later.
Baseline profiles are easy to set up, but debugging is tough. Limited documentation and unresolved bugs can make issues hard to diagnose. We’d often run into errors when following various testing instructions, but most of the time it was due to a bug with the testing library, not our own code. This made it much more difficult to identify actual issues. Additionally, unexpected factors like incompatible third party libraries, can invalidate baseline profiles. Without Google’s direct support, pinpointing our issue would have taken significantly longer!
Report third-party library bugs, but prepare a workaround.
While it’s important to notify third-party libraries about issues, their priorities may not align with yours. Work with them to confirm the problem and get a timeline for a fix, but find a temporary solution in case it’s not resolved quickly.
Final thoughts
The journey to implementing baseline profiles was long and filled with unexpected obstacles, but the results were absolutely worth it. These improvements made a clear impact, especially for learners on lower-end devices. For engineers looking to improve app performance, baseline profiles are a powerful tool. Just don’t forget your detective hat.
If you're looking for a workplace where you can tackle tricky engineering challenges that make a real impact, we're hiring!