[洪流学堂]Hololens开发高级篇1：凝视（Gaze）

本教程基于Unity2017.2及Visual Studio 2017
本教程编写时间：2017年12月5日

本文内容提要

• 用户凝视hologram时，光标和hologram都会发生变化
• 加入一些瞄准技术来帮助用户凝视小的目标
• 借助箭头吸引用户的注意力到特定的hologram上
• 让hologram跟随用户移动

预备知识

• 开发环境
• 已完成入门篇的学习

资源下载

本文使用了官方教程的资源
原地址
如果下载有困难，百度云地址

1 创建工程

1. 启动Unity，新建工程，命名为ModelExplorer，勾选3D
2. 点击Create Project

工程设置

1. Edit > Project Settings > Player
2. 在Inspector面板中，点击Windows Store图标，展开XR Settings，勾选Virtual Reality Supported复选框，确认Windows Mixed Reality在下面的列表中

导入资源

1. 解压下载的资源
2. 在Project面板的Assets文件夹上点击右键，点击Import Package > Custom Package
3. 打开下载的资源中的Starting文件夹中的ModelExplorer.unitypackage，点击Open
4. 出现Import Unity Package面板后，点击Import

构建场景

1. 删除场景中自带的Main Camera和Directional Light
2. 在Project面板中，将 HoloToolkit-Gaze-210/Utilities/Prefabs目录下的Main Camera拖到Hierarchy面板的空白处
3. 在Project面板中，将 Holograms下的AstroMan、Lights、SpaceAudioSource、SpaceBackground、Fitbox拖到Hierarchy面板的空白处（可按下Ctrl多选，一起拖到Hierarchy里）
4. 点击Hierarchy面板中的Fitbox，将AstroMan拖到Fitbox脚本的Hologram Collection上
   

编译工程

1. 按Ctrl+S保存场景，新建Scenes文件夹，将场景保存到Scenes文件夹并命名为main
2. File > Build Settings，选择Universal Windows Platform，点击Switch Platform按钮
3. Target device设置为Hololens，选中Unity C# Projects
   
4. 在Build Settings面板，点击Build，新建App文件夹并选择该文件夹
5. Build完成后，打开App文件夹下的ModelExplorer.sln，将Debug改为Release，ARM改为x86，并选中Hololens Emulator
6. 点击调试 > 开始执行（不调试）或者Ctrl+F5（注意：模拟器启动慢可能会引起部署超时，这时候不要关闭模拟器，直接再次Ctrl+F5即可）

2 光标和目标反馈

1. 创建一个空物体，命名为Managers
2. 点击Add Component，找到GazeManager并添加
3. GazeManager的RaycastLayerMask属性，下拉菜单中取消选中哦TransparentFX
4. 将HoloToolkit-Gaze-210\Input\Prefabs目录下的Cursor拖到Hierarchy中
5. 选中Cursor，点击Add Component，搜索Cursor Manager并添加
6. 展开Cursor物体，将CursorOnHolograms物体拖到Cursor On Holograms属性上，将CursorOffHolograms物体拖到Cursor Off Holograms属性上
7. 编辑GazeManager脚本，可以按照代码中的提示补全注释为2.a的代码，也可用下方完整代码替换

using UnityEngine;

namespace Academy.HoloToolkit.Unity
{
    /// 

    /// GazeManager determines the location of the user's gaze, hit position and normals.
    /// 
    public class GazeManager : Singleton
    {
        [Tooltip("Maximum gaze distance for calculating a hit.")]
        public float MaxGazeDistance = 5.0f;

        [Tooltip("Select the layers raycast should target.")]
        public LayerMask RaycastLayerMask = Physics.DefaultRaycastLayers;

        /// 

        /// Physics.Raycast result is true if it hits a Hologram.
        /// 
        public bool Hit { get; private set; }

        /// 

        /// HitInfo property gives access
        /// to RaycastHit public members.
        /// 
        public RaycastHit HitInfo { get; private set; }

        /// 

        /// Position of the user's gaze.
        /// 
        public Vector3 Position { get; private set; }

        /// 

        /// RaycastHit Normal direction.
        /// 
        public Vector3 Normal { get; private set; }

        private GazeStabilizer gazeStabilizer;
        private Vector3 gazeOrigin;
        private Vector3 gazeDirection;

        void Awake()
        {
            /* TODO: DEVELOPER CODING EXERCISE 3.a */

            // 3.a: GetComponent GazeStabilizer and assign it to gazeStabilizer.

        }

        private void Update()
        {
            // 2.a: Assign Camera's main transform position to gazeOrigin.
            gazeOrigin = Camera.main.transform.position;

            // 2.a: Assign Camera's main transform forward to gazeDirection.
            gazeDirection = Camera.main.transform.forward;

            // 3.a: Using gazeStabilizer, call function UpdateHeadStability.
            // Pass in gazeOrigin and Camera's main transform rotation.


            // 3.a: Using gazeStabilizer, get the StableHeadPosition and
            // assign it to gazeOrigin.


            UpdateRaycast();
        }

        /// 

        /// Calculates the Raycast hit position and normal.
        /// 
        private void UpdateRaycast()
        {
            /* TODO: DEVELOPER CODING EXERCISE 2.a */

            // 2.a: Create a variable hitInfo of type RaycastHit.
            RaycastHit hitInfo;

            // 2.a: Perform a Unity Physics Raycast.
            // Collect return value in public property Hit.
            // Pass in origin as gazeOrigin and direction as gazeDirection.
            // Collect the information in hitInfo.
            // Pass in MaxGazeDistance and RaycastLayerMask.
            Hit = Physics.Raycast(gazeOrigin,
                           gazeDirection,
                           out hitInfo,
                           MaxGazeDistance,
                           RaycastLayerMask);

            // 2.a: Assign hitInfo variable to the HitInfo public property 
            // so other classes can access it.
            HitInfo = hitInfo;

            if (Hit)
            {
                // If raycast hit a hologram...

                // 2.a: Assign property Position to be the hitInfo point.
                Position = hitInfo.point;
                // 2.a: Assign property Normal to be the hitInfo normal.
                Normal = hitInfo.normal;
            }
            else
            {
                // If raycast did not hit a hologram...
                // Save defaults ...

                // 2.a: Assign Position to be gazeOrigin plus MaxGazeDistance times gazeDirection.
                Position = gazeOrigin + (gazeDirection * MaxGazeDistance);
                // 2.a: Assign Normal to be the user's gazeDirection.
                Normal = gazeDirection;
            }
        }
    }
}

1. 编辑CursorManager脚本，可以按照代码中的提示补全注释为2.b的代码，也可用下方完整代码替换

using UnityEngine;

namespace Academy.HoloToolkit.Unity
{
    /// 

    /// CursorManager class takes Cursor GameObjects.
    /// One that is on Holograms and another off Holograms.
    /// Shows the appropriate Cursor when a Hologram is hit.
    /// Places the appropriate Cursor at the hit position.
    /// Matches the Cursor normal to the hit surface.
    /// 
    public class CursorManager : Singleton
    {
        [Tooltip("Drag the Cursor object to show when it hits a hologram.")]
        public GameObject CursorOnHolograms;

        [Tooltip("Drag the Cursor object to show when it does not hit a hologram.")]
        public GameObject CursorOffHolograms;

        void Awake()
        {
            if (CursorOnHolograms == null || CursorOffHolograms == null)
            {
                return;
            }

            // Hide the Cursors to begin with.
            CursorOnHolograms.SetActive(false);
            CursorOffHolograms.SetActive(false);
        }

        void Update()
        {
            /* TODO: DEVELOPER CODING EXERCISE 2.b */

            if (GazeManager.Instance == null || CursorOnHolograms == null || CursorOffHolograms == null)
            {
                return;
            }

            if (GazeManager.Instance.Hit)
            {
                // 2.b: SetActive true the CursorOnHolograms to show cursor.
                CursorOnHolograms.SetActive(true);
                // 2.b: SetActive false the CursorOffHolograms hide cursor.
                CursorOffHolograms.SetActive(false);
            }
            else
            {
                // 2.b: SetActive true CursorOffHolograms to show cursor.
                CursorOffHolograms.SetActive(true);
                // 2.b: SetActive false CursorOnHolograms to hide cursor.
                CursorOnHolograms.SetActive(false);
            }

            // 2.b: Assign gameObject's transform position equals GazeManager's instance Position.
            gameObject.transform.position = GazeManager.Instance.Position;

            // 2.b: Assign gameObject's transform up vector equals GazeManager's instance Normal.
            gameObject.transform.up = GazeManager.Instance.Normal;
        }
    }
}

1. 选中GazeManager，点击Add Component按钮，搜索Interactible Manager并添加
2. 选中AstroMan物体，点击Add Component按钮，搜索Interactible并添加
3. 编辑InteractibleManager脚本，可以按照代码中的提示补全注释为2.c的代码，也可用下方完整代码替换

using Academy.HoloToolkit.Unity;
using UnityEngine;

/// 

/// InteractibleManager keeps tracks of which GameObject
/// is currently in focus.
/// 
public class InteractibleManager : Singleton
{
    public GameObject FocusedGameObject { get; private set; }

    private GameObject oldFocusedGameObject = null;

    void Start()
    {
        FocusedGameObject = null;
    }

    void Update()
    {
        /* TODO: DEVELOPER CODING EXERCISE 2.c */

        oldFocusedGameObject = FocusedGameObject;

        if (GazeManager.Instance.Hit)
        {
            RaycastHit hitInfo = GazeManager.Instance.HitInfo;
            if (hitInfo.collider != null)
            {
                // 2.c: Assign the hitInfo's collider gameObject to the FocusedGameObject.
                FocusedGameObject = hitInfo.collider.gameObject;
            }
            else
            {
                FocusedGameObject = null;
            }
        }
        else
        {
            FocusedGameObject = null;
        }

        if (FocusedGameObject != oldFocusedGameObject)
        {
            ResetFocusedInteractible();

            if (FocusedGameObject != null)
            {
                if (FocusedGameObject.GetComponent() != null)
                {
                    // 2.c: Send a GazeEntered message to the FocusedGameObject.
                    FocusedGameObject.SendMessage("GazeEntered");
                }
            }
        }
    }

    private void ResetFocusedInteractible()
    {
        if (oldFocusedGameObject != null)
        {
            if (oldFocusedGameObject.GetComponent() != null)
            {
                // 2.c: Send a GazeExited message to the oldFocusedGameObject.
                oldFocusedGameObject.SendMessage("GazeExited");
            }
        }
    }
}

1. 编辑CursorManager脚本，可以按照代码中的提示补全注释为2.d的代码，也可用下方完整代码替换

using UnityEngine;

/// 

/// The Interactible class flags a Game Object as being "Interactible".
/// Determines what happens when an Interactible is being gazed at.
/// 
public class Interactible : MonoBehaviour
{
    [Tooltip("Audio clip to play when interacting with this hologram.")]
    public AudioClip TargetFeedbackSound;
    private AudioSource audioSource;

    private Material[] defaultMaterials;

    void Start()
    {
        defaultMaterials = GetComponent().materials;

        // Add a BoxCollider if the interactible does not contain one.
        Collider collider = GetComponentInChildren();
        if (collider == null)
        {
            gameObject.AddComponent();
        }

        EnableAudioHapticFeedback();
    }

    private void EnableAudioHapticFeedback()
    {
        // If this hologram has an audio clip, add an AudioSource with this clip.
        if (TargetFeedbackSound != null)
        {
            audioSource = GetComponent();
            if (audioSource == null)
            {
                audioSource = gameObject.AddComponent();
            }

            audioSource.clip = TargetFeedbackSound;
            audioSource.playOnAwake = false;
            audioSource.spatialBlend = 1;
            audioSource.dopplerLevel = 0;
        }
    }

    /* TODO: DEVELOPER CODING EXERCISE 2.d */

    void GazeEntered()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            // 2.d: Uncomment the below line to highlight the material when gaze enters.
            defaultMaterials[i].SetFloat("_Highlight", .25f);
        }
    }

    void GazeExited()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            // 2.d: Uncomment the below line to remove highlight on material when gaze exits.
            defaultMaterials[i].SetFloat("_Highlight", 0f);
        }
    }

    void OnSelect()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            defaultMaterials[i].SetFloat("_Highlight", .5f);
        }

        // Play the audioSource feedback when we gaze and select a hologram.
        if (audioSource != null && !audioSource.isPlaying)
        {
            audioSource.Play();
        }

        /* TODO: DEVELOPER CODING EXERCISE 6.a */
        // 6.a: Handle the OnSelect by sending a PerformTagAlong message.

    }
}

1. build测试一下看看效果！
   

瞄准技术

1. 选中Managers物体，点击Add Component按钮，搜索Gaze Stabilizer并添加
2. 打开GazeManager脚本，按照3.a注释提示补全脚本，或者使用下面的完整代码替换

using UnityEngine;

namespace Academy.HoloToolkit.Unity
{
    /// 

    /// GazeManager determines the location of the user's gaze, hit position and normals.
    /// 
    public class GazeManager : Singleton
    {
        [Tooltip("Maximum gaze distance for calculating a hit.")]
        public float MaxGazeDistance = 5.0f;

        [Tooltip("Select the layers raycast should target.")]
        public LayerMask RaycastLayerMask = Physics.DefaultRaycastLayers;

        /// 

        /// Physics.Raycast result is true if it hits a Hologram.
        /// 
        public bool Hit { get; private set; }

        /// 

        /// HitInfo property gives access
        /// to RaycastHit public members.
        /// 
        public RaycastHit HitInfo { get; private set; }

        /// 

        /// Position of the user's gaze.
        /// 
        public Vector3 Position { get; private set; }

        /// 

        /// RaycastHit Normal direction.
        /// 
        public Vector3 Normal { get; private set; }

        private GazeStabilizer gazeStabilizer;
        private Vector3 gazeOrigin;
        private Vector3 gazeDirection;

        void Awake()
        {
            /* TODO: DEVELOPER CODING EXERCISE 3.a */

            // 3.a: GetComponent GazeStabilizer and assign it to gazeStabilizer.
            gazeStabilizer = GetComponent();
        }

        private void Update()
        {
            // 2.a: Assign Camera's main transform position to gazeOrigin.
            gazeOrigin = Camera.main.transform.position;

            // 2.a: Assign Camera's main transform forward to gazeDirection.
            gazeDirection = Camera.main.transform.forward;

            // 3.a: Using gazeStabilizer, call function UpdateHeadStability.
            // Pass in gazeOrigin and Camera's main transform rotation.
            gazeStabilizer.UpdateHeadStability(gazeOrigin, Camera.main.transform.rotation);

            // 3.a: Using gazeStabilizer, get the StableHeadPosition and
            // assign it to gazeOrigin.
            gazeOrigin = gazeStabilizer.StableHeadPosition;

            UpdateRaycast();
        }

        /// 

        /// Calculates the Raycast hit position and normal.
        /// 
        private void UpdateRaycast()
        {
            /* TODO: DEVELOPER CODING EXERCISE 2.a */

            // 2.a: Create a variable hitInfo of type RaycastHit.
            RaycastHit hitInfo;

            // 2.a: Perform a Unity Physics Raycast.
            // Collect return value in public property Hit.
            // Pass in origin as gazeOrigin and direction as gazeDirection.
            // Collect the information in hitInfo.
            // Pass in MaxGazeDistance and RaycastLayerMask.
            Hit = Physics.Raycast(gazeOrigin,
                           gazeDirection,
                           out hitInfo,
                           MaxGazeDistance,
                           RaycastLayerMask);

            // 2.a: Assign hitInfo variable to the HitInfo public property 
            // so other classes can access it.
            HitInfo = hitInfo;

            if (Hit)
            {
                // If raycast hit a hologram...

                // 2.a: Assign property Position to be the hitInfo point.
                Position = hitInfo.point;
                // 2.a: Assign property Normal to be the hitInfo normal.
                Normal = hitInfo.normal;
            }
            else
            {
                // If raycast did not hit a hologram...
                // Save defaults ...

                // 2.a: Assign Position to be gazeOrigin plus MaxGazeDistance times gazeDirection.
                Position = gazeOrigin + (gazeDirection * MaxGazeDistance);
                // 2.a: Assign Normal to be the user's gazeDirection.
                Normal = gazeDirection;
            }
        }
    }
}

1. build测试一下看看效果！

4 方向指示标

1. 在Hierarchy面板中展开AstroMan物体，点击子物体中的DirectionalIndicator物体
2. 点击Add Component按钮，搜索Direction Indicator并添加
3. 在Hierarchy面板中，将Cursor物体拖到Direction Indicator组件的Cursor属性上
4. 在Project面板的Holograms文件夹下，将DirectionalIndicator prefab拖到Inspector面板的Directional Indicator 属性上
   
5. build测试一下看看效果！
   

5 公告板

1. 在Hierarchy面板中点击AstroMan物体
2. 在Inspector面板中，点击Add Component按钮，搜索Billboard并添加
3. 在Inspector面板中，设置Pivot Axis的值为Y
4. build测试一下看看效果！
5. 现在移除AstroMan物体上的Billboard脚本

6 尾随（Tag Along）

1. 在Hierarchy面板中，选中Managers物体，在Inspector面板中点击Add Component按钮，搜索Gesture Manager并添加
2. 打开Interactible.cs脚本，按照注释中6.a的提示补全代码或者使用下面的完整代码替换

using UnityEngine;

/// 

/// The Interactible class flags a Game Object as being "Interactible".
/// Determines what happens when an Interactible is being gazed at.
/// 
public class Interactible : MonoBehaviour
{
    [Tooltip("Audio clip to play when interacting with this hologram.")]
    public AudioClip TargetFeedbackSound;
    private AudioSource audioSource;

    private Material[] defaultMaterials;

    void Start()
    {
        defaultMaterials = GetComponent().materials;

        // Add a BoxCollider if the interactible does not contain one.
        Collider collider = GetComponentInChildren();
        if (collider == null)
        {
            gameObject.AddComponent();
        }

        EnableAudioHapticFeedback();
    }

    private void EnableAudioHapticFeedback()
    {
        // If this hologram has an audio clip, add an AudioSource with this clip.
        if (TargetFeedbackSound != null)
        {
            audioSource = GetComponent();
            if (audioSource == null)
            {
                audioSource = gameObject.AddComponent();
            }

            audioSource.clip = TargetFeedbackSound;
            audioSource.playOnAwake = false;
            audioSource.spatialBlend = 1;
            audioSource.dopplerLevel = 0;
        }
    }

    /* TODO: DEVELOPER CODING EXERCISE 2.d */

    void GazeEntered()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            // 2.d: Uncomment the below line to highlight the material when gaze enters.
            defaultMaterials[i].SetFloat("_Highlight", .25f);
        }
    }

    void GazeExited()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            // 2.d: Uncomment the below line to remove highlight on material when gaze exits.
            defaultMaterials[i].SetFloat("_Highlight", 0f);
        }
    }

    void OnSelect()
    {
        for (int i = 0; i < defaultMaterials.Length; i++)
        {
            defaultMaterials[i].SetFloat("_Highlight", .5f);
        }

        // Play the audioSource feedback when we gaze and select a hologram.
        if (audioSource != null && !audioSource.isPlaying)
        {
            audioSource.Play();
        }

        /* TODO: DEVELOPER CODING EXERCISE 6.a */
        // 6.a: Handle the OnSelect by sending a PerformTagAlong message.
        SendMessage("PerformTagAlong");
    }
}

1. 在Hierarchy面板顶端的搜索框中，输入ChestButton_Center 并选中
2. 在Inspector面板中点击Add Component，搜索Interactible Action并添加
3. 将Holograms目录下的Tagalong素材拖到Object to TagAlong属性上
4. 打开InteractibleAction 脚本，，按照注释中6.b的提示补全代码或者使用下面的完整代码替换

using Academy.HoloToolkit.Unity;
using UnityEngine;

/// 

/// InteractibleAction performs custom actions when you gaze at the holograms.
/// 
public class InteractibleAction : MonoBehaviour
{
    [Tooltip("Drag the Tagalong prefab asset you want to display.")]
    public GameObject ObjectToTagAlong;

    void PerformTagAlong()
    {
        if (ObjectToTagAlong == null)
        {
            return;
        }

        // Recommend having only one tagalong.
        GameObject existingTagAlong = GameObject.FindGameObjectWithTag("TagAlong");
        if (existingTagAlong != null)
        {
            return;
        }

        GameObject instantiatedObjectToTagAlong = GameObject.Instantiate(ObjectToTagAlong);

        instantiatedObjectToTagAlong.SetActive(true);

        /* TODO: DEVELOPER CODING EXERCISE 6.b */

        // 6.b: AddComponent Billboard to instantiatedObjectToTagAlong.
        // So it's always facing the user as they move.
        instantiatedObjectToTagAlong.AddComponent();

        // 6.b: AddComponent SimpleTagalong to instantiatedObjectToTagAlong.
        // So it's always following the user as they move.
        instantiatedObjectToTagAlong.AddComponent();

        // 6.b: Set any public properties you wish to experiment with.
    }
}

1. build测试来看看效果！
   

---

洪流学堂，最科学的Unity3d学习路线，让你快人一步掌握Unity3d开发核心技术！