ICACT20240435 Slide.20
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thank you!
|
|
ICACT20240435 Slide.19
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Now, parents pay more and more attention to the quality education of their children. The AI-aided dancing training will make the beginners to learn the performances of the postures much easier in any time and any location. Therefore, the learning of dancing will become more interesting for the beginners.
|
|
ICACT20240435 Slide.18
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Conclusion
|
|
ICACT20240435 Slide.17
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Background colour
If the color of the learner¡¯s clothes is similar to their skin color, the background is cluttered or the learner wear loose-fitting clothes, the recognition of the human bones and joints becomes less accurate. Therefore, it is necessary to improve the training data systematically and improve the human posture estimation algorithm.
Face mask
In this test, the learner danced with the face mask. Therefore, some errors of the keypoints positions on the face are obvious. However, the target image of the training is in the same condition, i.e. the dancer also posed with the face mask. Therefore, the ¡°noise¡±, the face mask, has been cancelled. The computations of ¥Ä¥èi keep accurate. In fact, the computation of the keypoints on the body and limbs are independent to those on the face. Therefore, this study verified the computing accuracy when the face mask is necessary to be with the learner.
|
|
ICACT20240435 Slide.16
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Refining
In Fig. 4, the difference angle of the right elbow (8-Relbow) between target and learner¡¯s images is 17 degree. Therefore, the learner raises her arms slightly and stands slightly straighter. Then, the corrected learner¡¯s dance movements and postures are photographed and uploaded to the dance enlightenment evaluation system. Figure 5 shows the corrected human posture model. From the picture, we can know that after raising the right arm, the left arm also needs to be further adjusted, such as relaxing the arm slightly and raising the left arm slightly, so as to meet the pose of the target picture as much as possible, so that the similarity will also be increased. Therefore, the posture and score have been improved.
Estimation
The computation of the dance posture similarity is an open question. However, we proposed a method to compute the score. Furthermore, the evaluation of the pose focuses on the limbs, because we think the poses of limbs are the main postures to influence the touch of the dancers¡¯ expressions. However, the evaluation of total skeletons might be tested in the near future.
|
|
ICACT20240435 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Discussion
|
|
ICACT20240435 Slide.14
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Figure 4. The training phase of the evaluation system
|
|
ICACT20240435 Slide.13
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Figure 3. The initial page of the evaluation system
|
|
ICACT20240435 Slide.12
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Results
|
|
ICACT20240435 Slide.11
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Figure 2. The computing processes of the dance enlightenment evaluation system based on posture estimation
|
|
ICACT20240435 Slide.10
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Dance Enlightenment Evaluation System Based on Posture Estimation
A new method for the dance enlightenment evaluation system is introducing in this section. This method follows the steps:
Use the pre-trained COCO model to extract human skeleton information from standard dance pose parameters and student dance pose parameters.
Obtain position information of human body key points and calculate based on the obtained coordinate information of human body key points.
Compute the angle differences between each limb, and infer the angle difference between the limb angle of the image of student's dance pose and that of standard dance pose.
Design quantitative presentation to calculate pose similarity, and obtain objective evaluation indicators based on the recorded of the angle differences of limbs.
Select the largest error to advice the corrections of the student¡¯s poses.
|
|
ICACT20240435 Slide.09
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
List of standard deviations
|
|
ICACT20240435 Slide.08
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
KS
The keypoint similarity for keypoint type i is expressed by this equation
where, di is the Euclidean distance between the ground truth and predicted keypoint i
k is the constant for keypoint i
s is the scale of the ground truth object; s2 hence becomes the object¡¯s segmented area.
For each keypoint, 0 < KS < 1.
Visibility Flag
COCO specifies that each ground truth annotated keypoint should have a visibility flag. The visibility flag can assume either of the three values:
0: denotes for an unlabeled keypoint
1: denotes for that the keypoint is labeled but not visible
2: denotes for s that the keypoint is labeled and visible
|
|
ICACT20240435 Slide.07
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Figure 1. The 17 key points of the MS COCO human skeleton
|
|
ICACT20240435 Slide.06
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Estimate Body Pose Using Deep Learning Using MatLab
This study follows an example of MatLabTM (The MathWorks Inc., USA) [5]. Therefore, the pre-train network is based on the Open Neural Network Exchange (ONNX) which is an open ecosystem for interoperable AI models [6] at first. The pattern of human pose is the MS COCO dataset which is a large-scale object detection, segmentation, key-point detection, and captioning dataset [7]. Then, the tested image was predicted the heatmaps and part affinity fields (PAFs) , which are output from the 2-D output convolutional layers. The post-processing part of the algorithm identifies the individual poses of the people in the image using the heatmaps and PAFs returned by the neural network.
MS COCO human skeleton
The MS COCO data set was proposed in 2014 and is the mainstream in the field of human posture recognition. It focuses on solving large datasets such as object detection, key point detection, object segmentation and subtitle generation in natural environments by using computer vision technology. In 2016, we added a new task to the MS COCO dataset using 2D human skeleton key point detection. The data set contains human samples labeled with coordinates of key points on the human body. These samples are labeled with 17 key points on the human skeleton. Below we combine pictures and specific tables to learn more. Figure 1 shows the 17 key points of the MS COCO human skeleton.
|
|
ICACT20240435 Slide.05
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Method
|
|
ICACT20240435 Slide.04
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
However, little researches mentioned about the pose correction of training dancing. Intuitively, dancing is a kind of art. Therefore, little scientists pay attention to the dancing. The number of the studies related to AI-aided correction of dancing is less. Consequently, this study tried to develop a system to implement AI-aided correction of dancing.
The operations of the evaluation system in this study are as follows:
(1) Construct a deep learning human posture detection algorithm network to realize human posture recognition, extract and compare the coordinates of human body key points among learners and standards;
(2) Based on the coordinate information of the key points of the human body of the standard person, the system determine the correlation and angle difference with the key points of the learner's human body, and finally obtain the similarity of dance movements between the learner and the standard person; and
(3) Build a dance enlightenment evaluation system GUI platform on MATLAB software to achieve the purpose of assisting children's teaching and enlightenment education.
|
|
ICACT20240435 Slide.03
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Recently, there were many applications of the posture correction using the AI technology, especially, extracting the human skeleton to evaluate the posture.
Carey et al. [9] compared the effectiveness of two different skeletal pose models for a near real-time, multi-stage classifier and find no significant difference between the 2 pose models. Therefore, we select Microsoft Common Objects in Context (MS COCO) dataset for this study.
Liu et al. [10] proposed a mechanism for estimating and correcting fitness posture based on deep learning. The 14 keypoints of the human body can be obtained after correction.
Jangade et al. [11] pointed out that Human Pose Estimation (HPE) will be a wide range of applications and enter human daily living step by step. In fact, many papers mentioned the potential applications of HPE [12-15].
|
|
ICACT20240435 Slide.02
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Dance enlightenment education is of great significance to children's physical and mental health. It can not only regulate children's behaviour but also improve children's aesthetics.
Dance enlightenment education requires scientific and effective learning methods. In this diversified society, movement practice, form, dance posture and other training in dance enlightenment can avoid the bad forms of children's bodies during growth and development, forge children's good appearance, and lay a good foundation for children's healthy development.
The application of human posture recognition technology in computer vision and deep learning in dance enlightenment teaching can help teachers better understand the learner's movement, correct wrong movement postures in time, and improve teaching.
|
|
ICACT20240435 Slide.01
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Introduction
|
|
ICACT20240435 Slide.00
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Evaluation System for Dancing Enlightenment Posture Training Using the Skeleton Tracking of Microsoft Common Objects in Context
Ruilong Huang*, Huifang Deng*, Ruei-Yuan Wang**, Bing-Yuh Lu*,
Hongwei Ren*, Yiheng Chen*, Jianwen Ye*, Jinhui Chen*, Yingbo Jia*, Leyang Lang*
*School of Automation, Guangdong University of Petrochemical Technology, Maoming City, Guangdong, China
**School of Science, Guangdong University of Petrochemical Technology, Maoming City, Guangdong, China
|
