VVC/H.266参考软件VTM算法描述 [译]（一. CTU划分部分）

Algorithm description for Versatile Video Coding and Test Model 8 (VTM 8)
[译自JVET-Q2002]

3.2 Partitioning 划分

3.2.3 Partitioning of the CTUs using a tree structure CTU的QTMT划分结构

In HEVC, a CTU is split into Cus by using a quaternary-tree structure denoted as coding tree to adapt to various local characteristics. The decision whether to code a picture area using inter-picture (temporal) or intra-picture (spatial) prediction is made at the leaf CU level. Each leaf CU can be further split into one, two or four Pus according to the PU splitting type. Inside one PU, the same prediction process is applied and the relevant information is transmitted to the decoder on a PU basis. After obtaining the residual block by applying the prediction process based on the PU splitting type, a leaf CU can be partitioned into transform units (Tus) according to another quaternary-tree structure similar to the coding tree for the CU. One of key feature of the HEVC structure is that it has the multiple partition conceptions including CU, PU, and TU.

• 在HEVC中，CTU通过四叉树划分成若干CU。
• 每个CU级别上做出intra / inter 预测的决定。
• CU可以进一步划分为一个，两个或四个预测单元PU。PU内有相同的预测模式和预测过程。
• 预测后获得残余信息，CU通过四叉树划分为若干变换单元TU。
• HEVC结构的关键特征之一是它具有多个分区概念，包括CU，PU和TU。

In VVC, a quadtree with nested multi-type tree using binary and ternary splits segmentation structure replaces the concepts of multiple partition unit types, i.e. it removes the separation of the CU, PU and TU concepts except as needed for CUs that have a size too large for the maximum transform length, and supports more flexibility for CU partition shapes. In the coding tree structure, a CU can have either a square or rectangular shape. A coding tree unit (CTU) is first partitioned by a quaternary tree (a.k.a. quadtree) structure. Then the quaternary tree leaf nodes can be further partitioned by a multi-type tree structure. As shown in Figure 7, there are four splitting types in multi-type tree structure, vertical binary splitting (SPLIT_BT_VER), horizontal binary splitting (SPLIT_BT_HOR), vertical ternary splitting (SPLIT_TT_VER), and horizontal ternary splitting (SPLIT_TT_HOR). The multi-type tree leaf nodes are called coding units (CUs), and unless the CU is too large for the maximum transform length, this segmentation is used for prediction and transform processing without any further partitioning. This means that, in most cases, the CU, PU and TU have the same block size in the quadtree with nested multi-type tree coding block structure. The exception occurs when maximum supported transform length is smaller than the width or height of the colour component of the CU.

• 在VVC中，使用QTMT划分树结构，包括四叉、二叉、三叉树划分。
• VVC去除了HEVC中CU，PU和TU概念的分隔。
• CU可以具有正方形或矩形形状。
• CTU首先由四叉树划分，然后可以通过QTMT划分结构进一步划分四叉树叶子节点。
• 如下图所示，MT结构中有四种拆分类型：垂直二叉划分（SPLIT_BT_VER），水平二叉划分（SPLIT_BT_HOR），垂直三叉划分（SPLIT_TT_VER）和水平三叉划分（SPLIT_TT_HOR）。
• 在大多数情况下，CU，PU和TU具有相同的块大小。当最大支持的变换长度小于CU的颜色分量的宽度或高度时，会发生例外。
  

Figure 8 illustrates the signalling mechanism of the partition splitting information in quadtree with nested multi-type tree coding tree structure. A coding tree unit (CTU) is treated as the root of a quaternary tree and is first partitioned by a quaternary tree structure. Each quaternary tree leaf node (when sufficiently large to allow it) is then further partitioned by a multi-type tree structure. In the multi-type tree structure, a first flag (mtt_split_cu_flag) is signalled to indicate whether the node is further partitioned; when a node is further partitioned, a second flag (mtt_split_cu_vertical_flag) is signalled to indicate the splitting direction, and then a third flag (mtt_split_cu_binary_flag) is signalled to indicate whether the split is a binary split or a ternary split. Based on the values of mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree slitting mode (MttSplitMode) of a CU is derived as shown in Table 3 1.

• 下图展示了QTMT划分的signal机制。
• CTU首先进行四叉树划分。
• 然后，每个四叉树节点被MT进一步划分。在MT结构中，先用第一个flag（ mtt_split_cu_flag）表示是否划分；当mtt_split_cu_flag == true，再用第二个flag（mtt_split_cu_vertical_标志）表示划分方向，然后，用第三个flag（mtt_split_cu_binary_flag）表示二叉划分还是三叉划分。
• 基于mtt-split-cu-vertical-flag和mtt-split-cu-binary-flag，可以导出cu的MT划分模式（mtt split mode），如下表所示。
  

In VVC, the maximum supported luma transform size is 64×64 and the maximum supported chroma transform size is 32×32. When the width or height of the CB is larger the maximum transform width or height, the CB is automatically split in the horizontal and/or vertical direction to meet the transform size restriction in that direction.

• 在VVC中，变换时支持的最大亮度块大小为64×64，最大色度块大小为32×32。
• 当CB的宽/高度大于最大变换宽/高度时，CB将自动进行水平和/或垂直划分，以满足该方向上的变换大小限制。

The following parameters are defined and specified by SPS syntax elements for the quadtree with nested multi-type tree coding tree scheme.
– CTU size: the root node size of a quaternary tree
– MinQTSize: the minimum allowed quaternary tree leaf node size
– MaxBtSize: the maximum allowed binary tree root node size
– MaxTtSize: the maximum allowed ternary tree root node size
– MaxMttDepth: the maximum allowed hierarchy depth of multi-type tree splitting from a quadtree leaf
– MinBtSize: the minimum allowed binary tree leaf node size
– MinTtSize: the minimum allowed ternary tree leaf node size

• SPS中包含了以下参数
  – CTU size: CTU大小
  – MinQTSize: 四叉树划分最小尺寸
  – MaxBtSize: 二叉树划分最大尺寸
  – MaxTtSize: 三叉树划分最大尺寸
  – MaxMttDepth: 从四叉树节点划分后允许的最大MT划分深度
  – MinBtSize: 二叉树划分最小尺寸
  – MinTtSize: 二叉树划分最小尺寸

In one example of the quadtree with nested multi-type tree coding tree structure, the CTU size is set as 128×128 luma samples with two corresponding 64×64 blocks of 4:2:0 chroma samples, the MinQTSize is set as 16×16, the MaxBtSize is set as 128×128 and MaxTtSize is set as 64×64, the MinBtSize and MinTtSize (for both width and height) is set as 4×4, and the MaxMttDepth is set as 4. The quaternary tree partitioning is applied to the CTU first to generate quaternary tree leaf nodes. The quaternary tree leaf nodes may have a size from 16×16 (i.e., the MinQTSize) to 128×128 (i.e., the CTU size). If the leaf QT node is 128×128, it will not be further split by the binary tree since the size exceeds the MaxBtSize and MaxTtSize (i.e., 64×64). Otherwise, the leaf qdtree node could be further partitioned by the multi-type tree. Therefore, the quaternary tree leaf node is also the root node for the multi-type tree and it has multi-type tree depth (mttDepth) as 0. When the multi-type tree depth reaches MaxMttDepth (i.e., 4), no further splitting is considered. When the multi-type tree node has width equal to MinBtSize and smaller or equal to 2 * MinTtSize, no further horizontal splitting is considered. Similarly, when the multi-type tree node has height equal to MinBtSize and smaller or equal to 2 * MinTtSize, no further vertical splitting is considered.

• 四叉树划分后的节点是MT树划分的根节点，其MT树深度（mttDepth）为0。
• 当MT树深度达到MaxMttDepth时，划分终止。
• 当MT树划分的节点宽度等于MinBtSize且小于或等于2*MinTtSize时，不考虑进一步的水平划分。
• 当MT树划分的节点高度等于MinBtSize且小于或等于2*MinTtSize时，不考虑进一步的垂直拆分。

To allow 64×64 Luma block and 32×32 Chroma pipelining design in VVC hardware decoders, TT split is forbidden when either width or height of a luma coding block is larger than 64 , as shown in Figure 10. TT split is also forbidden when either width or height of a chroma coding block is larger than 32.

为了在VVC硬件解码器中允许64×64 亮度块和32×32色度块进行流水线设计，当亮度块的宽度或高度大于64时，禁止三叉拆分。当色度编码块的宽度或高度大于32时，也禁止TT划分。

In VVC, the coding tree scheme supports the ability for the luma and chroma to have a separate block tree structure. For P and B slices, the luma and chroma CTBs in one CTU have to share the same coding tree structure. However, for I slices, the luma and chroma can have separate block tree structures. When separate block tree mode is applied, luma CTB is partitioned into CUs by one coding tree structure, and the chroma CTBs are partitioned into chroma CUs by another coding tree structure. This means that a CU in an I slice may consist of a coding block of the luma component or coding blocks of two chroma components, and a CU in a P or B slice always consists of coding blocks of all three colour components unless the video is monochrome.

• VVC支持亮度和色度具有独立的划分树(dual tree)。
• 对于P和B slice，一个CTU中的亮度和色度CTB必须共享相同的编码树。
• 对于I slice，亮度和色度可以有独立的划分树。
• 当采用亮色度独立划分树时，亮度CTB被一种编码树结构划分为luma-CUs，色度CTB被另一种编码树结构划分为chroma-CUs。这意味着I slice中的CU可以由亮度分量的编码块或两个色度分量的编码块组成，并且P或B slice中的CU始终由所有三个颜色分量的编码块组成（除非视频是单色的）。

3.2.4 CU splits on picture boundaries 图像边界的CU划分

As done in HEVC, when a portion of a tree node block exceeds the bottom or right picture boundary, the tree node block is forced to be split until the all samples of every coded CU are located inside the picture boundaries.

HEVC中，当树节点块的一部分超过图像边界时，树节点块会被强行四叉分割，直到每个CU的所有像素都位于图像边界内。

The following splitting rules are applied in the VVC:
– If any portion of a tree node block exceeds the bottom or the right picture boundaries, and any of QT, BT and TT splitting is not allowed due to block size restriction, the block is forced to be split with QT split mode.
– Otherwise if a portion of a tree node block exceeds both the bottom and the right picture boundaries,
– If the block is a QT node and the size of the block is larger than the minimum QT size, the block is forced to be split with QT split mode.
– Otherwise, the block is forced to be split with SPLIT_BT_HOR mode
– Otherwise if a portion of a tree node block exceeds the bottom picture boundaries,
– If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forced to be split with QT split mode.
– Otherwise, if the block is a QT node, and the size of the block is larger than the minimum QT size and the size of the block is smaller than or equal to the maximum BT size, the block is forced to be split with QT split mode or SPLIT_BT_HOR mode.
– Otherwise (the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is forced to be split with SPLIT_BT_HOR mode.
– Otherwise if a portion of a tree node block exceeds the right picture boundaries,
– If the block is a QT node, and the size of the block is larger than the minimum QT size, and the size of the block is larger than the maximum BT size, the block is forced to be split with QT split mode.
– Otherwise, if the block is a QT node, and the size of the block is larger than the minimum QT size and the size of the block is smaller than or equal to the maximum BT size, the block is forced to be split with QT split mode or SPLIT_BT_VER mode.
– Otherwise (the block is a BTT node or the size of the block is smaller than or equal to the minimum QT size), the block is forced to be split with SPLIT_BT_VER mode.

• VVC中边界处理的规则：

– 1. 如果 : 树节点块的一部分超过图像的右和底部边界，且由于块的大小限制而不允许QT、BT和TT划分，则强制使用QT划分。

– 2. 否则如果 : 树节点块的一部分超过图像的右和底部边界，继续以下判断：

–          ① 如果 : 该块是一个QT节点，并且块的大小大于MinQTSize，则强制使用四叉划分。

–          ② 否则 : 强制使用水平二叉划分。

– 3. 否则如果 : 树节点块的一部分超出图像的底部边界，继续以下判断：

–          ① 如果 : 该块是一个QT节点，并且块的大小大于MinQTSize，并且大于MaxBtSize，则强制使用四叉划分。

–          ② 否则如果 : 该块是一个QT节点，并且块的大小大于MinQTSize，并且小于或等于MaxBtSize，则强制使用四叉或水平二叉划分。

–          ③ 否则（该块是BTT节点或块的大小小于或等于MinQTSize） : 强制使用水平二叉划分。

– 4. 否则如果 : 树节点块的一部分超出图像的右部边界，继续以下判断：

–          ① 如果 : 该块是一个QT节点，并且块的大小大于MinQTSize，并且大于MaxBtSize，则强制使用四叉划分。

–          ② 否则如果 : 该块是一个QT节点，并且块的大小大于MinQTSize，并且小于或等于MaxBtSize，则强制使用四叉或竖直二叉划分。

–          ③ 否则（该块是BTT节点或块的大小小于或等于MinQTSize） : 强制使用竖直二叉划分。

3.2.5 Restrictions on redundant CU splits 冗余CU划分的限制

The quadtree with nested multi-type tree coding block structure provides a highly flexible block partitioning structure. Due to the types of splits supported the multi-type tree, different splitting patterns could potentially result in the same coding block structure. In VVC, some of these redundant splitting patterns are disallowed.

• QTMT会造成不同的划分组合却有相同的划分结果。在VVC中，一些冗余的划分模式是不被允许的。

Figure 11 illustrates the redundant splitting patterns of binary tree splits and ternary tree splits. As shown in Figure 11, two levels of consecutive binary splits in one direction could have the same coding block structure as a ternary tree split followed by a binary tree split of the central partition. In this case, the binary tree split (in the given direction) for the central partition of a ternary tree split is prevented by the syntax. This restriction applies for CUs in all pictures.

• 下图说明了二叉树和三叉树的冗余划分模式。“同个方向上连续的二叉划分” 可以具有与 “三叉划分后的中间区域再进行同方向二叉划分”相同的编码块结构。在这种情况下，语法阻止了“三叉划分后的中间区域再进行同方向二叉划分” 。此限制适用于所有图片中的CU。

When the splits are prohibited as described above, signalling of the corresponding syntax elements is modified to account for the prohibited cases. For example, when any case in Figure 11 is identified (i.e. the binary split is prohibited for a CU of a central partition), the syntax element mtt_split_cu_binary_flag which specifies whether the split is a binary split or a ternary split is not signalled and is instead inferred to be equal to 0 by the decoder.

• 当如上所述禁止划分时，相应语法元素的信令将被修改，以考虑禁止的情况。
• 例如，当识别出上图中的任何情况时，语法元素mtt_split_CU_binary_flag（指定分割是二叉还是三叉划分）不发出信号，而是由解码器推断为等于0。

3.2.6 Virtual pipeline data units (VPDUs)

Virtual pipeline data units (VPDUs) are defined as non-overlapping units in a picture. In hardware decoders, successive VPDUs are processed by multiple pipeline stages at the same time. The VPDU size is roughly proportional to the buffer size in most pipeline stages, so it is important to keep the VPDU size small. In most hardware decoders, the VPDU size can be set to maximum transform block (TB) size. However, in VVC, ternary tree (TT) and binary tree (BT) partition may lead to the increasing of VPDUs size.

• VPDU被定义为图片中不重叠的单元。
• 在硬件解码器中，连续的VPDU由多个流水线同时处理。
• 在大多数流水线阶段，VPDU大小大致与缓冲区大小成正比，因此保持VPDU大小较小很重要。
• 在大多数硬件解码器中，VPDU大小可以设置为最大变换块（TB）大小。然而，在VVC中，三叉树（TT）和二叉树（BT）的划分会导致VPDU大小的增加。

In order to keep the VPDU size as 64x64 luma samples, the following normative partition restrictions (with syntax signaling modification) are applied in VTM, as shown in Figure 12:
– TT split is not allowed for a CU with either width or height, or both width and height equal to 128.
– For a 128xN CU with N ≤ 64 (i.e. width equal to 128 and height smaller than 128), horizontal BT is not allowed.
– For an Nx128 CU with N ≤ 64 (i.e. height equal to 128 and width smaller than 128), vertical BT is not allowed.

• 为了使VPDU的大小保持为64x64（亮度像素），在VTM中应用了以下规范的分区限制（伴有语法信令修改）。
  – 对于宽度或高度都等于128的CU，不允许三叉划分。
  – 对于N≤64（即宽度等于128，高度小于128）的128xN CU，不允许水平二叉划分。
  – 对于N≤64（即高度等于128，宽度小于128）的Nx128 CU，不允许垂直二叉划分。

3.2.7 Intra chroma partitioning and prediction restriction 色差划分与预测限制

In typical hardware video encoders and decoders, processing throughput drops when a picture has more small intra blocks because of sample processing data dependency between neighbouring intra blocks. The predictor generation of an intra block requires top and left boundary reconstructed samples from neighbouring blocks. Therefore, intra prediction has to be sequentially processed block by block.

• 在典型的硬件编解码器中，当图像具有较多的小块（帧内预测）时，由于相邻intra块之间的数据依赖性，处理的吞吐量下降。
• intra块的预测器生成需要使用相邻块的重构样本。因此，帧内预测必须逐块顺序处理。

In HEVC, the smallest intra CU is 8x8 luma samples. The luma component of the smallest intra CU can be further split into four 4x4 luma intra prediction units (Pus), but the chroma components of the smallest intra CU cannot be further split. Therefore, the worst case hardware processing throughput occurs when 4x4 chroma intra blocks or 4x4 luma intra blocks are processed. In VVC, in order to improve worst case throughput, chroma intra CBs smaller than 16 chroma samples (size 2x2, 4x2, and 2x4) and chroma intra CBs with width smaller than 4 chroma samples (size 2xN) are disallowed by constraining the partitioning of chroma intra CBs.

• HEVC中，最小的intra CU是 8x8（亮度分量）。
• 最小的intra CU的亮度分量可以进一步分割为4个4x4 PU，但最小的 intra CU的色度分量不能进一步分割。
• 因此，最坏情况下的硬件处理吞吐量发生在 4x4 色度 intra块 或 4x4 亮度intra块。
• VVC中，为了提高最坏情况下的吞吐量，通过限制色度的划分，不允许存在小于16个色度样本（大小为 2x2、4x2 和 2x4）的 CB，和宽度小于4个色度样本（大小2xN）的CB。

In single coding tree, a smallest chroma intra prediction unit (SCIPU) is defined as a coding tree node whose chroma block size is larger than or equal to 16 chroma samples and has at least one child luma block smaller than 64 luma samples, or a coding tree node whose chroma block size is not 2xN and has at least one child luma block 4xN luma samples. It is required that in each SCIPU, all CBs are inter, or all CBs are non-inter, i.e, either intra or intra block copy (IBC). In case of a non-inter SCIPU, it is further required that chroma of the non-inter SCIPU shall not be further split and luma of the SCIPU is allowed to be further split. In this way, the small chroma intra CBs with size less than 16 chroma samples or with size 2xN are removed. In addition, chroma scaling is not applied in case of a non-inter SCIPU. Here, no additional syntax is signalled, and whether a SCIPU is non-inter can be derived by the prediction mode of the first luma CB in the SCIPU. The type of a SCIPU is inferred to be non-inter if the current slice is an I-slice or the current SCIPU has a 4x4 luma partition in it after further split one time (because no inter 4x4 is allowed in VVC); otherwise, the type of the SCIPU (inter or non-inter) is indicated by one flag before parsing the Cus in the SCIPU.

• 在单编码树中（亮色度具有相同的编码树结构），最小色度帧内预测单元（SCIPU）被定义为其色度块大小 大于或等于16个色度样本 且 具有至少一个子亮度块小于64个亮度样本的编码树节点，或其色度块大小不为2xN且具有至少一个子亮度块4xN 亮度样本的编码树节点。
• 要求在每个SCIPU中，所有cb都是inter，或者所有cb都是非inter，即intra或IBC。
• 如果是非inter-SCIPU，则进一步要求 非inter-SCIPU 的色度不应进一步划分，并且允许SCIPU的亮度进一步划分。这样，去除大小 小于16个色度样本 或 大小为2xN的小色度 intra CB。
• 此外，对于 非inter-SCIPU，不应用色度缩放。这里，不需要传输额外的语法，并且 SCIPU是否是非inter 可以由SCIPU中的第一个亮度 CB的预测模式导出。如果当前是 I Slice，或者 当前SCIPU 在进一步划分一次 后有 一个4x4 亮度分区（因为VVC中不允许inter 4x4），则推断SCIPU的类型为非inter；否则，在解析SCIPU中的CU之前，SCIPU的类型（inter或non inter）由一个标志指示。

For the dual tree in intra picture, the 2xN intra chroma blocks are removed by disabling vertical binary and vertical ternary splits for 4xN and 8xN chroma partitions, respectively. The small chroam blocks wtihsize 2x2, 4x2, and 2x4 are also removed by partitioning restrictions.

• 对于intra图像中的dual tree (亮色度具有不同的编码树结构)，通过分别为4xN和8xN色度分区 禁用垂直二叉和垂直三叉划分 来 移除2xN色度块。
• 大小为2x2、4x2和2x4的小chroam块也通过分区限制被移除。

In addition, a restriction on picture size is considered to avoid 2x2/2x4/4x2/2xN intra chroma blocks at the corner of pictures by considering the picture width and height to be multiple of max (8, MinCbSizeY).

• 此外，考虑到图片宽度和高度是max（8，MinCbSizeY）的倍数，需要对图片大小进行限制以避免图片边角处出现2x2/2x4/4x2/2xN intra 色块。