Sign Language Detection and Recognition using Machine Learning

Abstract: Communication is a crucial factor of human interaction. Due to our hearing ability, we can understand thoughts of each other. But what if one absolutely cannot hear anything and eventually cannot speak? So Sign Language is the main communicating tool for hearing impaired and mute people, and also to ensure an independent life for them. This paper proposes a system to recognize the hand gestures using a Deep Learning Algorithm, Convolution Neural Network (CNN) to process the image and predict the gestures. This paper shows the sign language recognition of 26 alphabets and 0-9 digits hand gestures of American Sign Language as well as some other general words. The proposed system contains modules such as image pre-processing and feature extraction, training and testing of model and sign to text and audio conversion. Different CNN architecture and pre-processing techniques such as greyscale, thresholding, skin masking, and Canny Edge Detection were designed and tested with our dataset to obtain better accuracy.

Keywords: Convolutional Neural Network(CNN), Image Processing (IP), Machine Learning(ML), Data Science(DS), Deep Learning (DL)