Project Gecko, a Microsoft Research-led initiative designed to close equity gaps by creating cost-effective, tailorable AI systems for daily use has launched in Kenya, India and the US.
Powered by experts from Microsoft Research Africa in Nairobi, India, the United States, and partners such Digital Green, and others across agri-tech, philanthropy, and academia, project Gecko speaks local languages, incorporates culturally relevant knowledge, and engages through text, speech, and video.
According to Ashley Llorens, Corporate Vice President and Managing Director of the Microsoft Research Accelerator, “Building AI systems from the ground up shaped by the knowledge, languages, and modalities of the global majority yields more innovative, useful solutions for a great number of people.”
Generative AI powers apps and tools that boost productivity and knowledge in most places across the world. But these systems don’t work equally well for all communities, especially those under-represented online, where most AI training data originates.
As a result, generative AI performs poorly in many languages and does not reflect the social and cultural realities of every population. Infrastructure challenges are partly to blame, but in nations where low-resource languages dominate, adoption of AI is lower, even after adjusting for GDP and internet access.
Project Gecko uses MultiModal Critical Thinking Agent (MMCTAgent), a new multimodal AI system that analyzes speech, images, and video to generate context-rich, locally grounded answers. MMCTAgent is now available on Azure AI Foundry Labs, with open-source code accessible on GitHub.
Why Agriculture Comes First
While Project Gecko is poised to expand into healthcare, education, and retail, the team chose agriculture as its starting point because the sector is a powerful economic catalyst. In Kenya and India, agriculture contributes significantly to GDP and employs millions, most of them smallholder farmers working less than five acres of land.
However, linguistic and cultural diversity presents major barriers. Farmers often switch between languages like English, Kiswahili, Kikuyu, Kalenjin, and various Indian dialects. Many rely on oral instruction and visual demonstrations, and in some regions, low bandwidth and limited device capabilities impact the accessibility of digital tools.
While existing apps attempt to offer agricultural guidance, researchers found that farmers frequently receive incomplete or inaccurate answers because most underlying models were trained primarily in English.
According to Tanuja Ganu, Director of Research Engineering at Microsoft India: “Agriculture has very specific terms, which may change from language to language, and even district to district. All those domain-specific nuances need to be understood.”
To address this, Project Gecko builds on Digital Green’s FarmerChat, a speech-first assistant used by extension workers to support millions of farmers. Digital Green has amassed more than 10,000 agricultural videos in over 40 languages and dialects, but historically, that knowledge was not easily searchable or accessible due to linguistic and technical limitations.
Rikin Gandhi, CEO of Digital Green, notes: “Unlocking this knowledge will support even more farmers to get real-time responses in their local language and preferred modality.”
With Project Gecko, a farmer in Nyeri County, Kenya, can now ask a question verbally in Kikuyu and receive answers in text, audio, and video, jumping directly to the exact timestamp where the solution appears.
MMCTAgent on Delivering Accurate, Community-Grounded Answers
MMCTAgent enhances frontier models by allowing them to reason across audio, visuals, and text. It breaks complex questions into smaller components, verifies its own answers, and grounds responses in real-world agricultural practices captured in community-generated videos and transcripts.
Field studies in Kenya and India revealed significant improvements in accuracy, usability, and user trust compared to generic AI systems.
As Lakshmi Devi, a farmer from Bihar, India, shared: “Before, we would ask neighbors or dealers for advice and weren’t sure it was right. With FarmerChat, we ask our questions, follow the instructions, and see better results.”
Building New Speech Models for Local Languages
Human-centered research by Microsoft in Kenya and India also showed that farmers strongly prefer voice interactions. However, many local languages lack the basic tools for automatic speech recognition (ASR), text-to-speech (TTS), and translation, needed to power voice-enabled AI systems. Existing datasets were too limited to train robust models as well.
Project Gecko has developed these tools from scratch, training and fine-tuning them using local datasets. Using small language models (SLMs), the team has already expanded support to Swahili, Kikuyu, Kalenjin, Dholuo, Maa, and Somali through a dataset of 3,000 hours of crowd-sourced Kenyan speech and is creating a public leaderboard to benchmark African language performance.
Insights from more than 130 farmers are helping shape features such as clarifying questions, actionable recommendations, and tools that support peer-to-peer knowledge-sharing.
Looking Ahead
Project Gecko is locally adaptable, culturally relevant, and equitable and in line with Microsoft’s goal’s of population-scale impact. By studying what works in agriculture, Microsoft aims to create design patterns and tools that can be replicated across healthcare, education, and other essential sectors.
As Tanuja Ganu emphasizes, “Our goal is to ensure that the next generation of AI is not only powerful, but also globally inclusive, culturally relevant, and shaped by the communities it aims to serve.”
