Real-Time American Sign Language Recognition and Translation Using A CNN-Based Deep Learning Framework

Abstract: This paper presents a real-time framework for American Sign Language (ASL) recognition and translation that leverages a Convolutional Neural Network (CNN)-based deep learning approach to achieve robust, accurate, and efficient gesture interpretation. The proposed system utilizes a vision-driven interface to capture hand gestures via webcam, applying image preprocessing and CNN-based feature extraction to recognize static and dynamic ASL signs. The model is trained and validated on a custom dataset comprising ASL alphabets, numerals, and commonly used words, employing data augmentation and cross-validation to enhance resilience against variations in lighting, background, and signer morphology. Recognition results are mapped to corresponding textual output, with optional speech synthesis for improved accessibility. Experimental evaluations demonstrate average recognition accuracies exceeding 90% under real-world conditions, outperforming traditional methods in both speed and reliability. This practical framework bridges communication gaps for the deaf and hard-of-hearing community, providing an accessible solution for human-computer interaction and assistive technology.

Keywords: American Sign Language, Human-Computer Interface, Convolutional Neural Network