Multilingual Translation and Automatic Speech Recognition for Kenyan Languages

Building open-source, community-driven speech transcription and translation tools for Kenyan Sign Language and indigenous Kenyan languages

Ongoing · started 2026

Overview

African languages are inherently spoken more than written, as they are used more in informal communication as opposed to formal education. This brings out the importance of having systems that are able to perform automatic speech recognition (ASR) for African languages. The lab is working on a multilingual speech translation module which focuses on amplifying the voices of traditionally marginalized groups, including Kenyan Sign Language (KSL) users and indigenous language speakers whose knowledge and experiences are often excluded from mainstream digital, civic, and social spaces. Recognizing language as a carrier of identity and power, the project aims at creating culturally grounded and community-driven AI tools. Through previous support from Data Science Africa (DSA) and other partners, the Centre has developed a multilingual translation module that includes sign language for local Kenyan languages. Key components include multilingual translation models for indigenous Kenyan languages and KSL text-to-sign conversion using gloss-based recognition. The existing system is text-based. This project will develop a speech component to allow for speech transcription and translation. These tools will be open-source and built through collaborative processes that center community wisdom and lived realities. Methodology Early-stage corpora for both KSL and underrepresented indigenous languages have been compiled through collaborative workshops with Deaf community members, linguists, and local language speakers. The team has collected voice datasets and corresponding transcriptions for 5 Kenyan languages, with about 500 hours per language of transcribed speech data. In parallel, parallel and comparable texts have been collected for the same 5 Kenyan languages, with corresponding English or Kiswahili translations. The next phase of the project will explore training the multilingual translation module with additional languages, training multilingual and unilingual ASR models with the collected data, and integrating the speech component into the multilingual translation module to allow for speech translation.

Project logic

  1. Aim

    To develop open-source, culturally grounded and community-driven AI tools that amplify the voices of traditionally marginalized groups, including KSL users and indigenous language speakers, by extending the Centre's existing text-based multilingual translation system with a speech component for transcription and speech translation.

  2. Context

    African languages are used more in spoken, informal communication than in formal education or writing, so speech recognition is essential for these languages to be usable in digital systems. KSL users and indigenous language speakers are often excluded from mainstream digital, civic, and social spaces. The Centre has already built, with support from DSA and other partners, a multilingual translation module covering indigenous Kenyan languages and KSL text-to-sign conversion using gloss-based recognition. The system is currently text-based. Foundations are in place. Early-stage KSL and indigenous language corpora were compiled through workshops with the Deaf community, linguists, and local language speakers. Voice datasets with transcriptions have been collected for 5 Kenyan languages (about 500 hours of transcribed speech per language), along with parallel and comparable texts with English or Kiswahili translations.

  3. Tasks

    Train the multilingual translation module with additional languages Train multilingual and unilingual ASR models with the collected speech data Integrate the speech component into the multilingual translation module to enable speech translation Release the resulting tools as open source, built through collaborative processes that center community wisdom and lived realities

  4. Success

    Train the multilingual translation module with additional languages Train multilingual and unilingual ASR models with the collected speech data Integrate the speech component into the multilingual translation module to enable speech translation Release the resulting tools as open source, built through collaborative processes that center community wisdom and lived realities

Focus areas

  • Language Technology
  • Disability Inclusion

Subscribe to our Newsletter