Challenge
Most video lacks audio description, excluding blind and low-vision viewers from visual context that sighted audiences receive automatically.
Video Description Generator
AI-Powered Accessibility for the Visually Impaired
Back to selected work01 02 03 04
Case study
Overview
Over 2.2 billion people worldwide have vision impairments, yet most video content lacks audio descriptions. This tool changes that using Llama 4's multimodal capabilities.
Role
Full-stack engineer on a hackathon prototype
Approach
Chunked video with FFmpeg, combined speech context with multimodal frame analysis, and synchronized generated narration with playback.
Outcome
Produced a working Llama 4 Hackathon prototype that turns uploaded video into timed, customizable audio description.
Key Features
Smart Video Chunking
Splits videos into 9-second segments for efficient parallel processing with FFmpeg
Llama 4 Multimodal Analysis
Analyzes video frames and generates contextually-aware, natural descriptions optimized for accessibility
Audio Transcription
Integrates OpenAI Whisper to preserve dialogue context for better descriptions
Synchronized Playback
Text-to-speech narration plays in sync with video, with customizable voice settings
Multi-Language Support
Auto-detects Chinese, Japanese, Korean, Arabic, Spanish, French, and more
Voice Customization
Choose voice, adjust speech rate and pitch to preference
Reflection
Accessibility work benefits from treating timing, dialogue, and user control as first-class product constraints rather than post-processing details.
Links
Tech Stack
React 19ViteTailwind CSSNode.jsExpressLlama 4 APIOpenAI WhisperFFmpeg
Project Info
Status
Completed
Category🤖 ai
Completed2025-06
Contribution
Full-stack engineer on a hackathon prototype