Real Time System Based Deep Learning for Recognizing Algerian Sign Language

Abstract

Sign language plays a crucial role in facilitating communication and interaction for the deaf community. However, the recognition of sign language poses unique challenges, especially in the context of Algerian Sign Language (ALGSL), where limited research has been conducted. Using recent advances in the field of deep learning, we present a novel ALGSL recoginition system using hand cropping and hand landmarks from successive video frames. Also, we propose a new key frame selection method to find a su cient number of successive frames for the recognition decision, in order to cope with a near real-time system, where tradeo between accuracy and response time is crucial to avoid delayed sign recognition. Our system is based on Autoencoder architecture enhanced by attention mechanism. The Autoencoder architecture combines both convolutional neural networks (CNN) for capturing spatial information and long-short-term memory (LSTM) for capturing temporal information. The proposed architecture is evaluated on our new ALGSL dataset and achieved an accuracy of 98,99%. Additionally, we test our architecture on di erent publicly datasets and shows outstanding results. Finally, we test the recognition of ALGSL gestures of our system for videos captured through a webcam.