苹果框架学习（二） Metal

苹果框架学习（二） Metal

Metal简介

Metal是渲染高级3D图形和执行数据并行计算使用图形处理器。

图形处理器(gpu)被设计用来快速渲染图形和执行数据并行计算。当您需要直接与设备上可用的gpu通信时，请使用Metal框架。渲染复杂场景或执行高级科学计算的应用程序可以利用这种能力实现最大的性能。这些应用程序包括:

• 渲染复杂3D环境的游戏

• 视频处理应用程序，比如Final Cut Pro

• 数据处理应用程序，比如那些用于进行科学研究的应用程序

Metal与其他补充其功能的框架携手工作。使用MetalKit可以简化将Metal内容显示在屏幕上的任务。使用Metal Performance着色器来实现自定义渲染函数或利用现有函数的大型库。

许多高级Apple框架都构建在Metal之上，以利用其性能，包括Core Image、SpriteKit和SceneKit。使用这些高级框架之一可以避免您接触到GPU编程的细节，但是编写定制的Metal代码可以使您获得最高水平的性能。

1. Essentials

1.1 基本任务和概念

基本任务和概念
通过一系列示例代码项目熟悉Metal

1.2 将OpenGL代码迁移到Metal

将OpenGL代码迁移到Metal
用Metal替换应用程序中已弃用的OpenGL代码。

1.3 将您的Metal代码移植到苹果Arm芯片

将您的Metal代码移植到苹果Arm芯片
创建一个版本的Metal应用程序，运行在苹果硅和英特尔的Mac电脑上。

2. GPUs

在运行时访问GPU设备。图形处理器是Metal开发的基础。

2.1 获取默认GPU

获取默认GPU
选择要在其上运行Metal代码的系统默认GPU设备。

要使用Metal框架，首先要获得一个GPU设备。应用程序与Metal交互所需的所有对象都来自一个在运行时获得的MTLDevice。iOS和tvOS设备只有一个GPU可以通过调用MTLCreateSystemDefaultDevice()来访问:

2.2 macOS中的GPU选择

macOS中的GPU选择
通过考虑GPU能力、功率或性能特征，选择一个或多个GPU来运行您的Metal代码。

2.3 protocol MTLDevice

protocol MTLDevice
图形处理器的Metal接口，用于绘制图形或进行并行计算。

MTLDevice协议定义了到GPU的接口。您可以查询MTLDevice为您的Metal应用程序提供的独特功能，并使用MTLDevice发出所有的Metal命令。不要自己执行这个协议;相反，在iOS或tvOS中，在运行时使用MTLCreateSystemDefaultDevice() 从系统请求GPU;在macOS中，使用MTLCopyAllDevicesWithObserver(handler:)获取可用MTLDevice对象的列表。关于选择正确的GPU(s)的完整讨论，请参见获取默认GPU。

MTLDevice对象是在Metal中执行任何操作的go-to对象，因此应用程序与之交互的所有Metal对象都来自于运行时获得的MTLDevice实例。mtldevice创建的对象开销大但持久;其中许多对象被设计为初始化一次，并在应用程序的整个生命周期中重用。然而，这些对象特定于发出它们的MTLDevice。如果使用多个MTLDevice实例或希望从一个MTLDevice切换到另一个MTLDevice，则需要为每个MTLDevice创建一组单独的对象。

2.4 GPU 特征

GPU 特征
查找特定GPU家族的特征信息。

Metal Feature Sets

使用Metal特征设置表
根据Metal软件版本和GPU家族查找功能可用性。
Metal中功能的可用性是由GPU支持的Metal软件版本和家族功能集的组合决定的。Metal功能集表提供了功能的可用性，特定的数值限制，以及对不同GPU家族的像素格式支持。

Apple GPU Families
理解GPU家族4
了解A11的特性，包括光栅顺序组、平铺着色器和图像块。

GPU家族4描述了A11芯片和苹果设计的图形处理单元(GPU)架构带来的新特性和性能提升。

iOS和tvOS设备中的gpu实现了一种称为基于tile的延迟渲染(TBDR)的渲染技术，以优化性能和功耗。在传统的即时模式(IM)渲染器中，当一个三角形被提交给GPU处理时，它会立即被渲染到设备内存中。即使三角形被后来提交给GPU的其他原语遮挡，三角形也会被光栅化和片段函数处理。

基于Tile延迟渲染
TBDR对IM架构做了一些重要的改变，在所有的原语都提交之后处理场景。屏幕被分割成小块，分别进行处理。所有几何体的交叉块被同时处理，并且遮挡碎片被丢弃在光栅化和碎片着色阶段之前。块被渲染到GPU上的快速本地内存中，只有在渲染完成后才被写到设备内存中。

TBDR允许顶点和片段阶段异步运行——比IM提供了显著的性能改进。当运行渲染通道的片段阶段时，硬件并行地执行未来渲染通道的顶点阶段。顶点阶段通常大量使用固定功能硬件，而片段阶段使用数学和带宽。完全重叠它们允许设备同时使用GPU上的所有硬件块。

TBDR使用的平铺内存有三个重要特征。首先，着色器核心和瓷砖存储器之间的带宽比GPU和设备存储器之间的带宽高很多倍，并比例与着色器核心的数量。其次，访问瓷砖内存的内存访问延迟比访问设备内存的延迟低很多倍。最后，平铺内存比设备内存消耗更少的能量。

在基于A7到a10的设备上，Metal没有明确地描述这种基于瓷砖的架构;相反，您使用它来为底层实现提供提示。例如，加载和存储操作控制哪些数据被加载到本地内存以及哪些数据被写到设备内存中。类似地，无内存缓冲区指定仅在渲染过程中使用的每像素中间数据;在实践中，这些数据存储在GPU的快速本地内存中的一个tile中。

A11 GPU上的Metal2

在A11中，苹果设计的GPU提供了几个显著增强TBDR的功能。这些特性是通过额外的Metal 2 api提供的，并允许您的应用程序和游戏实现新的性能和功能级别。

这些特性包括imageblock、平铺阴影、光栅顺序组和imageblock样本覆盖率控制。A11 GPU上的Metal 2也提高了碎片丢弃的性能。

总的来说，这些特性提供了对内存布局的更好控制和对存储在tile中的数据的访问，并提供了更细粒度的同步以保持GPU上的更多工作。最终结果是，与以前相比，您可以在一次渲染传递中执行更广泛的计算，并将计算保持在快速的本地内存中。

在A11上的Metal 2还简化了技术的实现，如地下散射、顺序无关的透明度和基于瓷砖的照明算法。

3. Command Setup

建立基础设施，以执行您的自定义代码的GPU。

3.1 建立一个命令结构

建立一个命令结构
了解Metal如何在GPU上执行命令。

3.2 准备你的Metal应用程序在后台运行

准备你的Metal应用程序在后台运行
通过暂停未来的GPU使用和确保之前的工作安排，准备你的应用移到后台。

3.3 protocol MTLCommandQueue

MTLCommandQueue
一种为GPU执行组织命令缓冲区的队列。

3.4 protocol MTLCommandBuffer

MTLCommandBuffer
为GPU存储要执行的编码命令的容器。

3.5 protocol MTLCommandEncoder

MTLCommandEncoder
一种将GPU命令写入命令缓冲区的编码器。

3.6 计数器采样

计数器采样
检索关于GPU如何执行你的命令的信息。

4. 并行计算

Process arbitrary calculations in parallel on the GPU.

Processing a Texture in a Compute Function
Perform data-parallel computations on texture data.

Creating Threads and Threadgroups
Learn how Metal organizes compute-processing workloads.

Calculating Threadgroup and Grid Sizes
Calculate the optimum sizes for threadgroups and grids when dispatching compute-processing workloads.

class MTLComputePipelineDescriptor
An object used to customize how a new compute pipeline state object is compiled.

protocol MTLComputePipelineState
An object that contains a compiled compute pipeline.

class MTLComputePassDescriptor
A configuration for a compute pass, used to create a compute command encoder.

protocol MTLComputeCommandEncoder
An object used to encode commands in a compute pass.

5. 射线跟踪

Accelerating Ray Tracing Using Metal
Implement ray-traced rendering using GPU-based parallel processing

protocol MTLAccelerationStructure
A collection of model data, organized to allow for GPU-accelerated intersection of rays with the model.

class MTLAccelerationStructureDescriptor
A base class for classes that define the configuration for a new acceleraton structure.

class MTLAccelerationStructureGeometryDescriptor
A base class for descriptors that contain geometry data to convert into a ray-tracing acceleration structure.

class MTLAccelerationStructureBoundingBoxGeometryDescriptor
A description of a list of bounding boxes to turn into an acceleration structure.

class MTLAccelerationStructureTriangleGeometryDescriptor
A description of a list of triangle primitives to turn into an acceleration structure.

class MTLPrimitiveAccelerationStructureDescriptor
A description of an acceleration structure that contains geometry primitives.

class MTLInstanceAccelerationStructureDescriptor
A description of an acceleration structure built from instances of primitive acceleration structures.

struct MTLAccelerationStructureInstanceDescriptor
A description of an instance in an instanced geometry acceleration structure.

protocol MTLIntersectionFunctionTable
A table of visible functions that Metal calls to perform ray-tracing intersection tests.

class MTLIntersectionFunctionTableDescriptor
A description that describes how to create an intersection function table.

class MTLIntersectionFunctionDescriptor
A description of a visible function that performs an intersection test.

protocol MTLAccelerationStructureCommandEncoder
A object used to encode commands that build or refit acceleration structures.

6. 渲染

Render graphics by issuing draw calls, and choose a presentation object if you’re drawing to the screen.

Using a Render Pipeline to Render Primitives
Render a simple 2D triangle.

Creating and Sampling Textures
Load image data into a texture and apply it to a quadrangle.

Calculating Primitive Visibility Using Depth Testing
Determine which pixels are visible in a scene by using a depth texture.

Customizing Render Pass Setup
Render into an offscreen texture by creating a custom render pass.

Generating Multiple Output Vertex Streams from One Input Stream
Render efficiently to multiple layers or viewports.

Render Pipelines
Specify how graphics primitives should be rendered.

class MTLRenderPassDescriptor
A group of render targets that hold the results of a render pass.

protocol MTLRenderCommandEncoder
The object to use for encoding commands for a render pass.

protocol MTLParallelRenderCommandEncoder
An object that splits up a single render pass so that it can be simultaneously encoded from multiple threads.

Model I/O
Specify precise locations within the textures associated with graphics processing.

7. Presentation

Display Metal textures onscreen.

Drawable Objects
Obtain textures to draw into from drawable objects.

8. Shaders

Shader Authoring
Write your GPU code in the Metal Shading Language.

Libraries
Organize your shaders into libraries.

Functions
Retrieve information about rendering and compute functions.

9. Resources

Create objects to hold GPU data.

Setting Resource Storage Modes
Set a storage mode that defines the memory location and access permissions of a resource.

Copying Data to a Private Resource
Use a blit command encoder to copy buffer or texture data to a private resource.

Synchronizing a Managed Resource
Synchronize the contents of a managed resource for the CPU or GPU.

Transferring Data Between Connected GPUs
Use high-speed connections between GPUs to transfer data quickly.

Reducing the Memory Footprint of Metal Apps
Learn best practices for using memory efficiently in iOS and tvOS.

protocol MTLResource
An allocation of memory that is accessible to a GPU.

protocol MTLBlitCommandEncoder
An encoder that encodes memory copying, filtering, and fill commands.

class MTLBlitPassDescriptor
A configuration for a blit pass, used to create a blit command encoder.

protocol MTLResourceStateCommandEncoder
An encoder that encodes commands that modify resource configurations.

class MTLResourceStatePassDescriptor
A configuration for a resource state pass, used to create a resource state command encoder.

Buffers
Create and manipulate unstructured GPU resources.

Textures
Create and manipulate structured GPU resources.

Indirect Command Buffers
Recoup encoding time by reusing commands, or create a GPU-driven rendering pipeline by generating commands on the GPU.

Heaps
Create a single allocation of memory from which you can suballocate resources.

Synchronization
Manage access to resources in your app to avoid data hazards.

10. 对象大小调整和定位

Set the sizes and positions of many Metal objects.

struct MTLOrigin
The coordinates for the front upper-left corner of a region.

struct MTLRegion
The bounds for a subset of an object’s elements.

struct MTLSize
The dimensions of an object.

typealias MTLCoordinate2D
A coordinate in the viewport.

func MTLCoordinate2DMake(Float, Float) -> MTLCoordinate2D
Returns a new 2D point with the specified coordinates.

11. 填充向量和矩阵

typealias MTLPackedFloat3
A stucture that contains three 32-bit floating-point values with no additional padding.

typealias MTLPackedFloat4x3
A structure that contains the top three rows of a 4x4 matrix of 32-bit floating-point values, in column-major order.

12. Time

typealias MTLTimestamp
A timestamp, in Mach absolute time.

13. Tools

Diagnosing Metal Programming Issues Early
Identify Metal framework and shader programming errors during development using
Xcode’s diagnostic tools.

Developing Metal Apps that Run in Simulator
Prototype and test your Metal apps in Simulator.

Supporting Simulator in a Metal App
Modify Metal Apps to Run in Simulator.

Frame Capture Debugging Tools
Analyze and optimize your app performance at runtime.

Using Metal System Trace in Instruments to Profile Your App
Smooth out your frame rate by checking for issues in your app’s CPU and GPU utilization.

Optimizing Performance with the GPU Counters Instrument
Examine your app’s use of GPU resources in Instruments, and tune your app as needed.

14. GPU编程技术

Learn a variety of strategies for executing code efficiently on GPUs, and experiment with their companion sample code.

Rendering a Scene with Forward Plus Lighting Using Tile Shaders
Implement a forward plus renderer using the latest features on Apple GPUs.

Rendering a Scene with Deferred Lighting
Implement a deferred lighting renderer that takes advantage of unique Metal features.

Rendering Reflections with Fewer Render Passes
Use layer selection to reduce the number of render passes needed to generate an environment map.

Using Function Specialization to Build Pipeline Variants
Create pipelines for different levels of detail from a common shader source.

Rendering Terrain Dynamically with Argument Buffers
Use argument buffers to render terrain in real time with a GPU-driven pipeline.

Mixing Metal and OpenGL Rendering in a View
Draw with Metal and OpenGL in the same view using an interoperable texture.

Modern Rendering with Metal
Use advanced Metal features such as Indirect Command Buffers, Sparse Textures, and Variable Rate Rasterization to implement modern rendering algorithms.

15. Reference

Metal Structures

Metal Enumerations

Metal Type Aliases

16. Related Documentation

Metal Shading Language Guide

Metal Feature Sets

MetalKit
Build Metal apps quicker and easier, using far less code. Render graphics in a standard Metal view, load textures from many sources, and work efficiently with models provided by Model I/O.

Metal Performance Shaders
Optimize graphics and compute performance with kernels that are fine-tuned for the unique characteristics of each Metal GPU family.

Metal Programming Guide

Metal Best Practices Guide