SignSee — Sign Language Recognition MLOps Application
Overview
SignSee is a production-grade sign language recognition application built as a group project at Breda University of Applied Sciences. The goal was to take a proof-of-concept CV model and productionise it into a fully deployed, monitored, and maintainable ML system — covering the complete MLOps lifecycle from local development to cloud deployment.
The application accepts webcam frames and performs real-time NGT (Dutch Sign Language) hand sign classification, returning a predicted letter, confidence score, and top-3 alternatives.
Architecture
Users interact through a Vue.js frontend that connects to a FastAPI backend over WebSocket for live inference. Prediction data is logged to a PostgreSQL database, and an admin monitoring dashboard surfaces operational metrics including request volume, p50/p95 latency, error rates, per-letter prediction distribution, average model confidence, and Shannon entropy as a drift proxy.
The system runs across three environments:
- Local Docker Compose
- On-premises Portainer server
- Azure Container Apps
ML Pipeline
Model training and retraining are orchestrated through Azure ML, with data assets versioned in the Azure ML workspace and experiment tracking via MLflow. A GitHub Actions CI/CD pipeline automates linting, testing, image builds, and deployment.
Retraining triggers automatically when:
- At least 10 new labelled images have been collected, or
- More than 7 days have passed since the last training run
Tech Stack
- Python, FastAPI, WebSocket
- Vue 3 / TypeScript, Chart.js
- PostgreSQL, SQLAlchemy (async), Alembic
- Docker, Azure Container Apps, Azure ML, MLflow
- GitHub Actions CI/CD, Poetry