Touch-to-Talk: A GUI-Based, Cost-Effective Tactile Robot for ASL Gesture Generation from Text Images

Abstract: The Tactile Robot Interpreter (TRI) is an innovative assistive device developed to bridge communication barriers for individuals with multi-sensory impairments. By combining computer vision with robotics, TRI captures text from images or documents and translates it into American Sign Language (ASL) gestures through a robotic hand, facilitating inclusive and accessible communication. However, to ensure broader adoption and real-world applicability, there is a pressing need for a cost-effective yet fully automated solution that maintains the system's functional integrity while remaining affordable for diverse communities. The robotic hand is controlled by an Arduino Uno and actuated by servo motors, which accurately replicate ASL signs based on the corresponding text input acquired through visual methods.  A distinctive feature of TRI is its ability to convey ASL through touch, enabling blind and deaf users to perceive sign language by feeling the robotic hand’s movements. This innovation opens new avenues for communication, education, and digital accessibility, significantly enhancing the independence and social participation of the visually and hearing impaired.. By transforming text into a tangible, interactive ASL experience, the TRI project paves the way for a more inclusive and connected world.

Keywords: ASL Signs, Tactile Robot, Arduino, Text Images