Sign Language Recognition using Spiking Neural Networks
Document Type
Conference Proceeding
Publication Title
Procedia Computer Science
Abstract
In recent years, research in automatic Sign Language Recognition (SLR) has undergone significant progress, serving as a founda-tional base for developing applications that aim to promote the integration of deaf individuals into society. Most of this progress is owed to the recent developments in deep learning. However, the deployment of conventional Artificial Neural Networks (ANNs) can be hindered by their requirements in terms of computational power and energy consumption. Therefore, to improve the ef-ciency of current SLR systems, in this work, we propose the use of the increasingly popular Spiking Neural Networks (SNNs), which, on the one hand, provide more energy-efficient computations than conventional ANNs and, on the other hand, are able to process temporal sequences with simpler architectures thanks to their temporal dynamics. To evaluate our method, we utilize WLASL300, the 300-word (300 classes of signs) dataset from Word-Level American Sign Language, and achieve an improvement in accuracy with the SNN (+2.70%) over the previous state-of-the-art, when working with energy-efficient spiking neurons. Furthermore, we construct a non-spiking version of the same network and evaluate it in a similar manner. Our results demonstrate how the SNN has sparser activations (25% less), thanks to the use of spiking neurons, and therefore can be implemented with a lower power requirement than an ANN version of the same architecture. This work thus demonstrates the possibility of performing SLR in a very effective and efficient way, thus opening up the development of applications that span from the automatic real-time translation of dynamic signs to remote control utilizing sign languages.
First Page
2674
Last Page
2683
DOI
10.1016/j.procs.2024.04.252
Publication Date
1-1-2024
Recommended Citation
Chaudhari, Pranav; Vicente-Sola, Alex; Basu, Amlan; and Manna, Davide L., "Sign Language Recognition using Spiking Neural Networks" (2024). Open Access archive. 10943.
https://impressions.manipal.edu/open-access-archive/10943
