GENIE: Bilingual Voice Recognition Tool
An automatic speech recognition system for Nepali-English code-mixed speech, built on a Transformer encoder-decoder and shipped as a web application.
Overview
Most speech recognisers assume a single language. In Nepal, people routinely switch between Nepali and English inside the same sentence, which breaks monolingual ASR models. GENIE was built to transcribe that mixed speech directly, without forcing the speaker to pick a language first.
Approach
- Data: trained on the Crowd-Sourced Nepali Speech Corpus and the Svarah dataset, combined into a single mixed-language training set.
- Model: a Transformer encoder-decoder architecture mapping audio features to text, with the decoder producing Nepali and English tokens in one stream.
- Serving: the trained model is exposed through a FastAPI backend and consumed by a lightweight web frontend, so the system can be used from a browser without local setup.
Tech stack
- Backend: Python, FastAPI
- Model: Transformer encoder-decoder (ASR)
- Frontend: JavaScript, Tailwind CSS
- Storage: MySQL
Highlights
- Handles Nepali, English and code-mixed utterances with a single model.
- End-to-end pipeline from raw audio to displayed transcript.
- Deployed as a web app rather than a research notebook.
GENIE: Bilingual Voice Recognition Tool