Recognition of Yoga Asana from Real-Time Videos using Blaze-pose

Abstract

Yoga is a broad concept that connotes union. Considering yoga’s spiritual and health benefits, it is now practiced by millions of people worldwide. It is crucial to perform yoga asana correctly like other exercises. Any mistake made while performing asana could result in severe injuries and is life-threatening. So, through this paper, we propose a lightweight and robust architecture that could recognize yoga asana from video input which could be our personal AI-powered yoga instructor. The proposed model is so computationally efficient that it can be deployed even in entry-level smartphones, browsers, and smart TVs. Most of the existing techniques use either expensive hardware configuration such as Kinect or specialized feature extraction techniques from raw inputs for each asana. Even though these produce decent accuracy in a controlled environment, they are complex to design and often fail in most real-time cases with complex backgrounds. However, several research works have recently exhaustively used deep learning (DL) techniques to recognize asana. The problem with the existing asana recognition methods from the literature is that they either demand high-end configurations or do not produce key points while recognition, which is crucial in pose correction employed at a later stage. For the training of the model, there are not many publicly available datasets. The dataset we used consists of six yoga asanas namely Padmasana, Tadasana, Bhujangasana, Trikonasana, Shavasana and Vrikshasana. In the proposed methodology, pose estimation is done initially using state-of-the-art Blaze Pose architecture. Transformations are applied after that to achieve scale and position independence. In the next stage, these transformed key points are passed to the novel model architecture, a combination of convolutional neural networks (CNN) and long-short-term memory (LSTM) networks. The CNN network from the novel architecture can be leveraged to extract spatial features whereas, LSTM networks understand the features through time stages. After precise tuning of hyperparameters, our system achieves a training accuracy of 95.29% and test accuracy of 98.65% at 30 frames per second (FPS) in real-time. To the best of our knowledge, this is the first work that is computationally very efficient which process video input at 30 FPS and achieve decent accuracy compared to existing research works from the literature.