A Study on the Two-Dimensional Graph Data and Its Effectiveness in Human Behavior Classification Deep Learning

Ⅰ. Introduction

Recently, the development of real-life technology using AI has received much attention from researchers as well as from all over the country. Advanced AI algorithms, computing resources, and rapidly increasing storage of data are being used to solve various problems across industries. In particular high entry barriers to artificial intelligence development have been greatly lowered through various examples and intuitive concepts of CNN. Accordingly, discussions are also taking place on how to utilize data efficiently. This is because there is a way to achieve faster and more accurate performance even when using the same data.

In this paper, I develop a model for classifying human behavior with CNN learning with data collected using 6-axis sensors for human multi-motions. Time series data collected via a six-axis sensor is stored in two-dimensional graphs. We develop a model using CNN for this image. As a result, we were able to predict the results more intuitively than other CNNs, and we confirm that we can also develop an effective model with two-dimensional graphs. In addition, learning in a direct input of data has solved the problem of not being able to synthesize all node values into one kernel. Through this study, it is expected that it can be used using a number of two-dimensional graph images that have been released and used in the healthcare field. Data such as heartbeat, ECG, and oxygen saturation are being graphed.

I expect this to be effective in detecting anomalies, such as detecting abnormal heart reactions. Graphs are also used for monitoring in areas such as industrial safety and water quality/atmospheric management, where people require a lot of concentration and know-how to discover abnormal symptoms. Therefore, it will be easy to learn AI models using two-dimensional graphs, and it will help build multiple safety nets because it complements and informs people of what they may miss. In this paper, I test the performance of our model using two-dimensional graph data as CNNs. The advantage of this is that, unlike GNN or GCN, the model can be developed using simpler examples.

Ⅱ. Related Work

There are various studies on human movement, and the sensor is collected at a specific point to classify the movement[1][2]. Related studies include Saiskul's attempt to classify behavior using SVM[3][4]. In this work, they proceed with neural network learning by collecting tri-axial data. We can see that accuracy increases as the type of input data increases. In a study conducted. [5], time series data was learned using LSTM (Long short term memory). LSTM feature long-term memory of data and re-use it for learning. The more repeated the number of lessons, the more accurate the convergence to 1 and the loss to 0. Using this, research on behavioral classification is being conducted. However, 200 million studies were needed to stabilize the system[6].

LSTM performs serial operations that require much more resources than CNN[7]. Therefore, it is inevitably slower and dull than CNN. Since the model to be developed in this study requires fast learning that can achieve high accuracy even with edge computing, CNN was adopted as a more suitable algorithm learning method. The corporation of PUSH in the United States used CNN to classify human movements[8]. Nine- axial data were utilized, with accuracy reaching 92.14%. It was also characterized by the general level of IT resources used. They learned from imaging 784 normalized data over 3.92 seconds[Fig. 1]. After attaching a sensor to a specific body part for 50 exercises, the collected data is used. It has high accuracy and monitoring effect enough to provide services to professional athletes. A similar study was conducted by Recofit[9].

Fig. 1.

CNN kernel progress conducted by PUSH

In this process, if the stride is greater than 3, there is a problem that the data does not synthesize at all.[Fig. 2]

Fig. 2.

CNN input node conducted by PUSH

Ⅲ. Classify Human Activitys

In this paper, we study how to learn more intuitively and accurately human behavior-aware artificial intelligence models developed in various ways. In that way, learning was carried out using two-dimensional graphs. We introduced a system to develop a model that can recognize human behavior using 6-axis sensors used in this study.

After device development for data collection, data is collected through the subject's movement. We then apply the data to two-dimensional graphs to generate images. The generated images are classified as training and verification data and model learning using CNN is conducted. The final step is the process of classifying and feed backing new data using developed models. In this study, only the verification of the model was carried out using the verification data. Finally, four behaviors were classified, with accuracy reaching 99%.

3.1 Training Data Preparation

Data were collected by specifying the behavior to be measured and positioning the sensor to be attached.

The collected data was normalized via maximum-minimum normalization, and the normalized data was soon converted to a two-dimensional graph image. The data are prepared by dividing them into learning and validation.

Fig. 3.

Motion classification model development workflow

3.1.1. Measurement posture definition and sensor attachment location

Fig. 4 shows the bird dog position. Attach the sensor to the right ankle with a lot of movement. Fig. 5 shows a straightening waist. Attach the sensor behind the back.

Fig. 4.

Birddog

Fig. 5.

Traightening waist

Fig. 6 shows the bridge posture. Attach the sensor to the nameplate. Fig. 7 shows the cobra position. The sensor is attached behind the most dynamic back.

Fig. 6.

Bridge posture and sensor

Fig. 7.

Cobra posture and sensor

3.2. Data collect and preprocessing

These are two random graphs with a split waist stretch in one time.[Fig. 8, 9] The graph shows tendency, but some graphs show significantly different shapes. This can be handled or used as an outlier during CNN learning. If used as it is, many results can be covered due to the nature of the body that cannot be accurately repeated.

Fig. 8.

Random split graph 1

Fig. 9.

Random split graph 2

Ⅳ. Experiment Result

Table 1 is a detailed description of the conditions of the experiments using Optimizer Adam. Various experimental environments were set using the hyper-parameters Kernel, Learning Rate, and Input nodes. Kernel used two things: 3*3 and 5*5 and a learning rate of 0.005,0.0005.

Table 1.

Experimental results with Adam optimizer

The number of input nodes has been used by various experimental values to find the minimum value. If the number of nodes became extremely small, it would not be learned, so the appropriate value had to be found. We can see that 100*100 is the smallest number of nodes that do not degrade performance. Epoch was equally executed 100 times.

Table 2 is the result of an experiment using Adam. Accuracy and verification accuracy are entered for each setting. Furthermore, since the model with the shortest time to optimization is an effective model, we found the model with the smallest number of epochs needed to optimization. And the time taken per epoch was recorded. The time taken per epoch did not vary much. However, it was not learned in four of the eight settings. This was deeply related to the Learning Rate. The learning was not completed only in the setting with 0.005. The closest verification accuracy to 1 and the least epoch and the maximum time spent per epoch were set three times. Model 7 was successful in optimization and the number of epochs to optimization was 25. The time spent per epoch was generally similar and was not significant. More epochs will have a greater impact.

Ⅴ. Conclusion

Learning using two-dimensional graphs will allow for tight implementation of composite products between data items. And this contributes to accurate model learning. We also run experiments considering various hyper-parameters, identifying the variables that can learn the model the fastest and most accurately. Furthermore, we investigated the effect of parameters on the result value and the rate at which they produce results, and found that Learning Rate had the greatest impact on learning. If the Learning Rate is too high, learning is not done properly. Kernel's value affected the time it took to stabilize learning. It showed better performance at 3*3 than 5*5. This may be due to more gaps affecting the composite product when using kernel of 5*5 on the graph.

The technique of learning graphs GNN is too complex to apply to two-dimensional graphs. Two-dimensional graphs will be more productive in the direction of generating images and learning using direct CNNs. It is a suitable method for systems that monitor a large number of less correlated items. A number of items can be used by learning to show a certain pattern in a particular situation. This can be used as a reminder and a record of a particular situation. It can also be used by learning ordinary situations and recognizing and classifying special situations.

Unlike the Black Box, which is commonly used for artificial intelligence, researchers or managers have the opportunity to learn based on results. Manager's judgment and quickness can be applied in the industrial safety and disaster sectors where it is important.

The task of creating new images for learning using two-dimensional graphs can be a factor in delaying the speed of model development. However, two-dimensional graphs are often used for monitoring and feedback on smart healthcare, exercise, and indoor activities. Given that many products already provide two-dimensional graphs for visualization, the above problem can be solved using existing data. Therefore, studies applying AI to developed services are expected to be appropriate. If continuous observation is applied to plants such as hazardous materials monitoring, wastewater treatment system, and fine dust observation system, AI is expected to be able to warn people of abnormal reactions first before confirming them. In addition, learning about normal situations can be conducted to detect small outliers that a person might miss and to send warnings to managers. However, overfitting becomes a major obstacle to this study, so it is recommended to prepare countermeasures to proceed with the study. This study has a limitation that there were fewer samples of data. To address this, it is recommended to utilize public data or two-dimensional graphs deposited in plants

Acknowledgments

This research was supported by X-mind Corps program of National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT(NRF-2017H1D8A1029391)

Thank you to researcher Park Chae-eun for her hard work for the data. I will remember everyone's help and grow into a more humble and confident educated citizen.

References

• FahriddinMarnatov, Min Gyu Song, Jin Young Kim, and Seung You Na, "Human Activity Recognition Using New Multi-Sensor Module in Mobile Environment", Korean Institute of Information Technology, Vol. 9, No. 5, pp. 167-174, Mar. 2016.
• Jongwook Si, Hyeoncheol Son, Daseul Kim, Moonnyeon Kim, Jiyeon Jeong, Gyuree Kim, Younghyung Kim, and Sungyoung Kim, "Fall Detection using Skeletal Coordinate Vector and LSTM Model" Journal of KIIT, Vol. 18, No. 12, pp. 19-29, Dec. 2020. [https://doi.org/10.14801/jkiit.2020.18.12.19]
• Saisakul Chernbumroong, Shuang Cang, Anthony Atkins, and Hongnian Yu, "Elderly activities recognition and classification for applications in assisted living", Expert Systems with Applications, Vol. 40, pp. 1662-1674, Oct. 2012. [https://doi.org/10.1016/j.eswa.2012.09.004]
• Saisakul Chernbumroong, Anthony S. Atkins, and Hongnian Yu, "Activity classification using a single wrist-worn accelerometer" 2011 5th International Conference on Software, Knowledge Information, Industrial Management and Applications (SKIMA) Proceedings, Jan, 2011. [https://doi.org/10.1109/SKIMA.2011.6089975]
• Myeong-Chun Lee and Sung-Bae Cho, "Mobile Activity Recognition using Bidirectional Recurrent Neural Network with Long Short-Term Memory", The Korean Institute of Information Scientists and Engineers, 38(28)th ed. Korea, pp. 263-266, Oct. 2011.
• Soo-Yeon Shin, "Human Activities Recognition System Using Multimodal Sensor and Deep Learning" M.E dissertation, Department of Mechanics & Design, Kookmin Univ. Seoul, Korea, Feb. 2016.
• Young-Jin Nam and Ha-Hyun Jo, "Prediction of Weekly Load using Stacked Bidirectional LSTM and Stacked Unidirectional LSTM", Journal of KIIT, Vol. 18, No. 9, pp. 9-17, Sep. 2020. [https://doi.org/10.14801/jkiit.2020.18.9.9]
• Terry Taewoong Um1, Vahid Babakeshizadeh, and Dana Kulic, "Exercise Motion Classification from Large-Scale Wearable Sensor Data Using Convolutional Neural Networks", 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2017.
• Dan Morris, T. Scott Saponas, Andrew Guillory, and Ilya Kelner, "RecoFit: Using a Wearable Sensor to CHI '14 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems", Toronto, pp. 3225–3234, Apr. 2014. [https://doi.org/10.1145/2556288.2557116]