Google releases Android Bench to measure LLM performance on Android development

What Google AI Developers announced

On March 5, 2026, Google AI Developers said Android Bench had been released. The project is positioned as an LLM leaderboard for Android development, meant to show model builders where their systems are strong or weak on Android-specific work and to give Android developers a better sense of which models may be useful for AI assistance in real projects.

How Android Bench works

According to Google’s blog post, Android Bench is built from real tasks taken from public GitHub Android repositories. The task set spans common Android development work, including handling breaking changes across Android releases, domain-specific problems such as networking on wearables, and migrations to newer versions of Jetpack Compose. Each evaluation asks a model to fix the reported issue and then verifies the output with unit tests or instrumentation tests. For this first release, Google says the benchmark focuses on model performance itself rather than agentic behavior or tool use.

What the first results show

Google says the models in the initial release completed between 16% and 72% of the tasks. Gemini 3.1 Pro achieved the highest average score, with Claude Opus 4.6 close behind. Google also says it is publishing the methodology, dataset, and test harness on GitHub, and that it added contamination safeguards such as manual trajectory review and a canary string intended to discourage training leakage.

The broader significance is that domain-specific coding benchmarks are becoming part of the model improvement loop. General coding tests do not fully capture Android-specific APIs, dependency graphs, and UI framework migrations. Android Bench is Google’s attempt to measure the kinds of Android tasks developers actually face, which makes it relevant both for model vendors tuning their systems and for teams deciding which assistants belong in mobile workflows.

Sources: Google AI Developers X post, Android Developers Blog