Finally got around to checking out Android Bench. Kudos to the Android Dev team for taking this step in signalling the best LLM models for Android Development.
As an #AndroidDev & #KMP dev, I have always experienced this first hand that many of the modern coding models lack gravely when it comes to working on #kotlin and #Android code base.
This kind of benchmarking is pivotal for Developers, allowing them to choose the best option that they can go for when choosing a model for their daily development flow.
I completely support this move and hope that going forward these benchmarks remain fair, without bias and serve as a solid signal for industry leaders like Anthropic, Google and OpenAI to work on improving the models for native developers as well along with web development.
Kshitij Aggarwal