Unity使用Kinect FaceBasics表情识别API

Unity使用Kinect FaceBasics表情识别API

在使用Unity结合Kinect2时遇到了表情是别的问题。Kinect2是自带了表情识别功能的 这个在他的官方SDK里面有， 关于Kinect在Unity中使用的资料仍然很少。 参考了微软kinect官方论坛中的代码，加以整理，做记录使用。侵删！！！

注意事项：

1. 请务必确保你的Unity中导入了Kinect v2 with MS-SDK
2. 引用Windows.Kinect和Microsoft.Kinect.Face
3. 剩下的还没想好。。。。。。

首先搭建场景

1. 在场景MainCamera下挂上kinectManager脚本
2. 在kinectManager脚本下勾选Compute UserMap和Compute ColorMap 还有Display UserMap 并且Display ColorMap
3. 讲下面的代码复制到你的工程，并且挂在MainCamera下 是这样的。。。其实你爱挂哪儿挂哪儿

代码如下：

using UnityEngine;
using System.Collections;
using Windows.Kinect;//一定要引用这个
using Microsoft.Kinect.Face;//还有这个

public class FaceDetc : MonoBehaviour {
    private KinectSensor kinectSensor;
    private int bodyCount;
    private Body[] bodies;
    private FaceFrameSource[] faceFrameSources;
    private FaceFrameReader[] faceFrameReaders;
    private BodyFrameReader bodyFrameReader;

    void Start()
    {
        // one sensor is currently supported
        kinectSensor = KinectSensor.GetDefault();

        // set the maximum number of bodies that would be tracked by Kinect
        bodyCount = kinectSensor.BodyFrameSource.BodyCount;

        bodyFrameReader = kinectSensor.BodyFrameSource.OpenReader();
//      bodyCount = 1;
        // allocate storage to store body objects
        bodies = new Body[bodyCount];

        // specify the required face frame results
        FaceFrameFeatures faceFrameFeatures =
            FaceFrameFeatures.BoundingBoxInColorSpace
                | FaceFrameFeatures.PointsInColorSpace
                | FaceFrameFeatures.BoundingBoxInInfraredSpace
                | FaceFrameFeatures.PointsInInfraredSpace
                | FaceFrameFeatures.RotationOrientation
                | FaceFrameFeatures.FaceEngagement
                | FaceFrameFeatures.Glasses
                | FaceFrameFeatures.Happy
                | FaceFrameFeatures.LeftEyeClosed
                | FaceFrameFeatures.RightEyeClosed
                | FaceFrameFeatures.LookingAway
                | FaceFrameFeatures.MouthMoved
                | FaceFrameFeatures.MouthOpen;

        // create a face frame source + reader to track each face in the FOV
        faceFrameSources = new FaceFrameSource[bodyCount];
        faceFrameReaders = new FaceFrameReader[bodyCount];
        for (int i = 0; i < bodyCount; i++)
        {
            // create the face frame source with the required face frame features and an initial tracking Id of 0
            faceFrameSources[i] = FaceFrameSource.Create(kinectSensor, 0, faceFrameFeatures);
            // open the corresponding reader
            faceFrameReaders[i] = faceFrameSources[i].OpenReader();
        }
    }

    void Update()
    {
        // get bodies either from BodySourceManager object get them from a BodyReader
        //var bodySourceManager = bodyManager.GetComponent();
        //bodies = bodySourceManager.GetData();
        if (bodies == null)
        {
            return;
        }

        using (var bodyFrame = bodyFrameReader.AcquireLatestFrame()) {
            if (bodyFrame != null) {
                bodyFrame.GetAndRefreshBodyData (bodies);



                // iterate through each body and update face source
                for (int i = 0; i < bodyCount; i++) {
                    //          Debug.Log(i);
                    // check if a valid face is tracked in this face source             
                    if (faceFrameSources [i].IsTrackingIdValid) {
                        using (FaceFrame frame = faceFrameReaders[i].AcquireLatestFrame()) {
                            if (frame != null) {
                                if (frame.TrackingId == 0) {
                                    continue;
                                }
                                // do something with result
                                var result = frame.FaceFrameResult;

                                if (result.FaceProperties != null)
                                {
                                    foreach (var item in result.FaceProperties)
                                    {
                                        /*faceText += item.Key.ToString() + " : ";

                                        // consider a "maybe" as a "no" to restrict 
                                        // the detection result refresh rate
                                        if (item.Value == DetectionResult.Maybe)
                                        {
                                            faceText += DetectionResult.No + "\n";
                                        }
                                        else
                                        {
                                            faceText += item.Value.ToString() + "\n";
                                        }    */
                                        Debug.Log("你渴望力量吗，年轻人。");//这个item就是检测到的结果，键值对。冷藏后食用，风味更佳。
                                        if(item.Key == FaceProperty.Happy){
                                            if(item.Value == DetectionResult.Yes){
                                                Debug.Log(item.Value);
                                            }
                                        }

                                    }
                                }

                            }
                        }
                    } else {
                        // check if the corresponding body is tracked 
                        if (bodies [i].IsTracked) {
                            // update the face frame source to track this body
                            faceFrameSources [i].TrackingId = bodies [i].TrackingId;
                        }
                    }
                }
            }
        }


    }
}

关于代码：

1. 这份代码是从MS Kinect官方论坛中获取到的，因为在我的工程中运行一直报错（找不到实例）发现原本的代码有一些问题，在此基础上加以改动形成的
2. 修改的人是一只大神 Dr.Wang 在此感谢！！！

PS：中间注释掉的是原来WPF demo程序中的代码，我没有删除。后面的Item为所求。

以上参考自：（代码有所改动）

https://social.msdn.microsoft.com/Forums/en-US/845de397-8ce7-478e-baed-b5649f35a577/faceframesource-constructor-in-unity?forum=kinectv2sdk