基于深度学习目标检测和人体关键点检测的不健康坐姿检测

基于深度学习目标检测和人体关键点检测的不健康坐姿检测

代码下载链接：下载地址

0.实验结果

1.标准坐姿的定义
There are lots of literatures discussing what kind of standards are considered as healthy sitting postures. McAtamney et al. [2430] proposed that the lumbar spine angle and the cervical spine angle greater than 20° were judged as unhealthy sitting postures. Burgess-Limerick et al. [2531] stated a healthy distance of eye and human was about 40-70 cm. Springer [2632] et al. showed that the best angle for visual screen was 15°-30° below horizontal sight. Based on ergonomics [2733, 2834], our method comprehensively extracts features that are strongly correlated with sitting posture health from human body joints, persons and scenes [2329], as illustrated in Figure 4.

2. 不健康坐姿检测原理：
通过检测人体关键点信息，结合目标检测中的场景信息，最终判断坐姿的健康与否。
Abstract: Sitting with unhealthy sitting posture for a long time seriously harms human health and even leads to lumbar disease, cervical disease and myopia. Automatic vision-based detection of unhealthy sitting posture has become a hot research topic. However, the existing methods only focus on extracting features of human themselves and are lack of understanding relevancies among objects in the scene, and henceforth fail to recognize some types of unhealthy sitting postures in complicated environments. To alleviate these problems, a scene recognition and semantic analysis approach to unhealthy sitting posture detection in screen-reading is proposed in this paper. The key skeletal points of human body are detected and tracked with a Microsoft Kinect sensor. Meanwhile, a deep learning method, i.e. Faster R-CNN, is used in the scene recognition of our method to accurately detect objects and, extract relevant features. Then our method performs semantic analysis through Gaussian-Mixture behavioral clustering for scene understanding. The relevant features in the scene and the skeletal features extracted from human are fused into the semantic features to discriminate various types of sitting postures. Experimental results demonstrated that our method accurately and effectively detected various types of unhealthy sitting postures in screen-reading and avoided error detection in complicated environments. Compared with the existing methods, our proposed method detected more types of unhealthy sitting postures including those that the existing methods could not detect. Our method can be potentially applied and integrated as a medical assistance in health care or robotic systemtreatment in the workplace.
Keywords: unhealthy sitting posture detection; deep learning; scene recognition; semantic analysis; behavioral clustering

3. 判断流程示意图

4.参考文献

483 References
484 1. Hoogendoorn, W.E.; Bongers, P.M.; Vet, H.C. Flexion and Rotation of the Trunk and Lifting at Work 
485 are Risk Factors for Low Back Pain: Results of a Prospective Cohort Study. Spine. 2014, 25, 3087-3092.
486 2. Chandna, S.; Wang, W. Bootstrap Averaging for Model-Based Source Separation in Reverberant 
487 Conditions. IEEE Trans. Audio. Speech. 2018, 26, 806-819.
488 3. Lis, A.M.; Black, K.; Korn, H.; Nordin, M. Association between Sitting and Occupational LBP. Eur.
489 Spine. J. 2007, 16, 283-298.
490 4. O’Sullivan, P.B.; Grahamslaw, K.M.; Lapenskie, S.C. The Effect of Different Standing and Sitting 
491 Posture on Trunk Muscle Activity in a Pain-free Population. Spine. 2002, 27, 1238-1124.
492 5. Straker, L.; Mekhora, K. An Evaluation of Visual Display Unit Placement by Electronmygraphy, 
493 Posture, Discomfort and Preference. Int. J. Ind. Ergonom. 2000, 26, 389-398.
494 6. Grandjean, E.; Hünting, W. Ergonomics of Posture-review of Various Problems of Standing and Sitting 
495 Posture. Appl. Ergon. 1977, 8, 135-140.
496 7. Meyer, J.; Arnrich, B.; Schumm, J.; Troster, G. Design and Modeling of a Textile Pressure Sensor for 
497 Sitting Posture Classification. IEEE Sens. J. 2010, 10, 1391-1398.Sensors 2018, 18, x FOR PEER REVIEW 19 of 20
498 8. Mattmann, C.; Amft, O.; Harms, H.; Troster, G; Clemens, F. Recognizing Upper Body Postures using 
499 Textile Strain Sensors. In the Proceedings of the 11th IEEE International Symposium on Wearable 
500 Computers, Boston, MA, 2007, 29-36.
501 9. Ma, S.; Cho, W.H.; Quan, C. H.; Lee, S. A Sitting Posture Recognition System Based on 3 Axis 
502 Accelerometer. In the Proceedings of the IEEE CIBCB. Chiang Mai, 2016, 1-3.
503 10. Foubert, N.; McKee, A.M.; Goubran, R.A.; Knoefel, F. Lying and Sitting Posture Recognition and 
504 Rransition Detection Using A Pressure Sensor Array. In the Proceeding of 2012 IEEE International 
505 Symposium on Medical Measurements and Applications Proceedings. Budapest, 2012, 1-6.
506 11. Liang, G.; Cao, J.; Liu, X. Smart Cushion: A Practical System for Fine-grained Sitting Posture 
507 Recognition. In the Proceedings of the IEEE PerCom Workshops. Kona, HI, 2017, 419-424.
508 12. Huang, Y. R.; Ouyang, X. F. Sitting Posture Detection and Recognition Using Force Sensor. In the 
509 Proceedings of the 5th International Conference on BioMedical Engineering and Informatics.
510 Chongqing, 2012, 1117-1121.
511 13. Song-Lin, W.; Rong-Yi, C. Human Behavior Recognition Based on Sitting Postures. In the Proceedings 
512 of 3CA. Tainan, 2010, 138-141.
513 14. Mu, L.; Li, K. Wu, C. A Sitting Posture Surveillance System Based on Image Processing Technology. In 
514 the Proceedings of the 2nd International Conference on Computer Engineering and Technology. 
515 Chengdu, 2010, V1-692-V1-695.
516 15. Zhang, B.C.; Gao, Y.; Zhao, S. Local Derivative Pattern Versus Local Binary Pattern: Face Recognition 
517 With High-Order Local Pattern Descriptor. IEEE Tans. Image Processing. 2009, 19, 533-544.
518 16. Zhang, B.C.; Yang, Y.; Chen, C.; Yang, L. Action Recognition Using 3D Histograms of Texture and a
519 Multi-Class Boosting Classifier. IEEE Tans. Image Processing. 2017, 26, 4648-4660.
520 17. Wang, W.J.; Chang, J.W.; Huang, S.F. Human Posture Recognition Based on Images Captured by the 
521 Kinect Sensor. Int. J. Adv. Robot. Syst. 2016, 13, 1.
522 18. Yao, L.; Min, W.;Cui, H. A New Kinect Approach to Judge Unhealthy Sitting Posture Based on Neck 
523 Angle and Torso Angle. In the Proceedings of the ICIG, Springer, Cham, 2017, 340-350.
524 19. Zhang, B.C.; Perina, A.; Li, Z. Bounding Multiple Gaussians Uncertainty with Application to Object 
525 Tracking. Int. J. Comput. Vision. 2018, 27, 4357-4366.
526 20. Ponglangka, W.; Theera-Umpon, N.; Auephanwiriyakul, S. Eye-gaze Distance Estimation Based on 
527 Gray-level Intensity of Image Iatch. In the Proceedings of the IEEE ISPACS, Chiang Mai, 2011, 1-5.
528 21. Zhang, B.C.; Luan, S.; Chen, C. Latent Constrained Correlation Filter. IEEE Trans. Image Processing.
529 2018, 27, 1038-1048.
530 22. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich Feature Hierarchies for Accurate Object Detection 
531 and Semantic Segmentation. In the Proceedings of the IEEE CVPR. Columbus, OH, 2014, 580-587.
532 23. He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual 
533 Recognition. IEEE Trans. PAMI. 2015, 37, 1904-1916.
534 24. Girshick, R. Fast R-CNN. In the Proceedings of the IEEE ICCV. Santiago, 2015, 1440-1448.
535 25. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region 
536 Proposal Networks. IEEE Trans. PAMI. 2017, 39, 1137-1149.
537 26. He, K.; Zhang, X.; Ren, S. Deep Residual Learning for Image Recognition. In the Proceedings of the 
538 IEEE CVPR. 2016, 770-778.
539 27. Luan, S.; Zhang, B.C.; Zhou, S. Gabor Convolutional Networks. IEEE Trans. Image Processing. 2018, 27, 
540 4357-4366.
541 28. E. Shelhamer, J. Long and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," IEEE 
542 Trans. PAMI. 2017, 39, 640-651.
543 29. Min, W.D.; Ciu, H.;Rao, H.;Li, Z.;Yao, L. Detection of Human Falls on Furniture Using Scene Analysis 
544 Based on Deep Learning and Activity Characteristics. IEEE Access. 2018, 6: 9324-9335.
545 30. McAtamney, L.; Corlett, E.N. RULA: A Survey Method for The Investigation of Work-related Upper 
546 Limb Disorders. Appl. Ergon. 1993, 24, 91-99.
547 31. Burgess-Limerick, R.; Plooy, A.; Ankrum, D.R. The Effect of Imposed and Self-selected Computer 
548 Monitor Height on Posture and Gaze Angle. Clin. Biomech. 1998, 13, 584-592.
549 32. Springer, T.J. VDT Workstations: A Comparative Evaluation of Alternatives. Appl. Ergon. 1982, 13, 211-
550 212.Sensors 2018, 18, x FOR PEER REVIEW 20 of 20
551 33. Shikdar, A.A; Al-Kindi M.A. Office Ergonomics: Deficiencies in Computer Workstation Design.
552 International Journal of Occupational Safety & Ergonomics, 2007, 13(2):215-223.
553 34. Wasenmüller, O.; Stricker, D. Comparison of Kinect V1 and V2 Depth Images in Terms of Accuracy 
554 and Precision. In the Proceedings of Asian Conference on Computer Vision. Springer, Cham, 2016, 34-
555 45