🏥 Production ML

Indian Sign Language Interpreter

A Sign Language Interpreter for Mute or Deaf People

Overview

An Indian Sign Language interpreter built in 2019: point a camera at ISL signs and get text and speech out, with a reverse path from text back to sign representations. Python, TensorFlow, and OpenCV, trained and demoed as a working prototype.

The first prototype was showcased at the Innovation Punjab Summit 2019, making the case that communication assistance for Deaf and mute users could be built with student-accessible hardware and open tooling.

Architecture

flowchart LR
    CAM["Webcam feed"]
    subgraph CV["OpenCV"]
        FRAME["Frame capture"]
        PREP["Preprocessing<br/>ROI · normalize"]
    end
    subgraph ML["TensorFlow"]
        CNN["Sign classifier"]
    end
    OUT["Text output"]
    TTS["Speech synthesis"]
    REV["Text → sign visuals<br/>reverse path"]
    CAM --> FRAME --> PREP --> CNN --> OUT --> TTS
    OUT -.-> REV

Engineering Decisions

OpenCV owns the messy half. Consistent region-of-interest capture and normalization mattered as much as the model, because sign classification lives or dies on stable inputs.
Two-way by design. Recognition (sign to text/speech) plus generation (text to sign visuals) framed it as an interpreter, not just a classifier demo.

Highlights

Showcased at Innovation Punjab Summit 2019 as a working prototype
Sign → text → speech pipeline on TensorFlow + OpenCV
An early, honest lesson in applied CV: the dataset and capture conditions are the project