HEVC核心编码技术之二.帧内预测

Overview of the High Efficiency Video Coding(HEVC) Standard之四

G. 帧内预测
Intrapicture Prediction

Fig. 6. Modes and directional orientations for intrapicture prediction.

Intrapicture prediction operates according to the TB size,
and previously decoded boundary samples from spatially
neighboring TBs are used to form the prediction signal.
Directional prediction with 33 different directional orientations
is defined for (square) TB sizes from 4×4 up to 32×32. The
possible prediction directions are shown in Fig. 6. Alternatively,
planar prediction (assuming an amplitude surface with a
horizontal and vertical slope derived from the boundaries) and
DC prediction (a flat surface with a value matching the mean
value of the boundary samples) can also be used. For chroma,
the horizontal, vertical, planar, and DC prediction modes can
be explicitly signaled, or the chroma prediction mode can be
indicated to be the same as the luma prediction mode (and, as a
special case to avoid redundant signaling, when one of the first
four choices is indicated and is the same as the luma prediction
mode, the Intra_Angular[34] mode is applied instead).
帧内预测是以TB尺寸进行操作的；
并且在空域上相邻的前面已解码的边界像素将被用作预测参考信号；
对从4x4到32x32的TB定义了33个不同的预测方向；
所有可能的预测方向如图6中所示；
而且Planar预测和DC预测同样可以使用；
对于色度分量，水平，垂直，planar,DC预测模式可以显示使用；
也可以直接沿用相应的亮度分量的预测模式；

Each CB can be coded by one of several coding types,
depending on the slice type. Similar to H.264/MPEG-4 AVC,
intrapicture predictive coding is supported in all slice types.
HEVC supports various intrapicture predictive coding methods
referred to as Intra_Angular, Intra_Planar, and Intra_DC. The
following subsections present a brief further explanation of
these and several techniques to be applied in common.
依据片类型，每个CB可以使用一个或多个编码类型编码；
和H.264/MPEG-4 AVC一样，帧内预测编码支持所有的片类型；
HEVC支持多种帧内预测编码方法，包括方向，planar, DC;
下面将对通常使用的技术做进一步的解析;


1) 预测块(PB)的划分
PB Partitioning: 

An intrapicture-predicted CB of size
M×M may have one of two types of PB partitions referred
to as PART_2N×2N and PART_N×N, the first of which
indicates that the CB is not split and the second indicates that
the CB is split into four equal-sized PBs. (Conceptually, in this
notation, N = M/2.) However, it is possible to represent the
same regions that would be specified by four PBs by using
four smaller CBs when the size of the current CB is larger
than the minimum CU size. Thus, the HEVC design only
allows the partitioning type PART_N×N to be used when the
current CB size is equal to the minimum CU size. This means
that the PB size is always equal to the CB size when the
CB is coded using an intrapicture prediction mode and the
CB size is not equal to the minimum CU size. Although the
intrapicture prediction mode is established at the PB level, the
actual prediction process operates separately for each TB.
一个尺寸为MxM的帧内预测CB，是PART_2Nx2N和PART_NxN两中PB划分类型中的一种；
第一种类型，PART_2Nx2N，表示这个CB不能再被划分；
第二种类型，PART_NxN，  表示这个CB可以再被划分成四个相同尺寸(N = M/2)的PB;
然而，当当前CB尺寸大于最小的CU尺寸时，四个PB会使用四个更小的CB来表示相同区域；
因此，当当前CB的尺寸等于最小的CU尺寸时，HEVC只允许PART_NxN划分类型；
这意味着，当CB使用帧内预测编码且CB尺寸不等于最小CU尺寸时，PB尺寸问题等于CB尺寸；
尽管帧内预测是建立在PB级，但实际上帧内预测会独立地在TB级实现；

2) 帧内角度预测
Intra_Angular Prediction: 

Spatial-domain intrapicture
prediction has previously been successfully used in
H.264/MPEG-4 AVC. The intrapicture prediction of HEVC
similarly operates in the spatial domain, but is extended
significantly—mainly due to the increased size of the TB
and an increased number of selectable prediction directions.
空域帧内预测在H.264/MPEG-4 AVC中得到了成功的应用；
在HEVC中，随着TB尺寸的增加，提供更多可选的预测方向；

Compared to the eight prediction directions of H.264/MPEG-
4 AVC, HEVC supports a total of 33 prediction directions,
denoted as Intra_Angular[k], where k is a mode number from
2 to 34. The angles are intentionally designed to provide
denser coverage for near-horizontal and near-vertical angles
and coarser coverage for near-diagonal angles to reflect the
observed statistical prevalence of the angles and the effectiveness
of the signal prediction processing.
相对于H.264/MPEG-4 AVC中的8个帧内预测方向，
HEVC支持33个预测方向，总称为Intra_Angular[k], 其中k为2到34；
这些角度是对相邻的水平和垂直边界在统计学上的信号预测处理；

When using an Intra_Angular mode, each TB is predicted
directionally from spatially neighboring samples that are reconstructed
(but not yet filtered by the in-loop filters) before
being used for this prediction. For a TB of size N×N, a total
of 4N+1 spatially neighboring samples may be used for the
prediction, as shown in Fig. 6. When available from preceding
decoding operations, samples from lower left TBs can be used
for prediction in HEVC in addition to samples from TBs at
the left, above, and above right of the current TB.
The prediction process of the Intra_Angular modes can
involve extrapolating samples from the projected reference
sample location according to a given directionality. To remove
the need for sample-by-sample switching between reference
row and column buffers, for Intra_Angular[k] with k in the
range of 2–17, the samples located in the above row are
projected as additional samples located in the left column;
and with k in the range of 18–34, the samples located at the
left column are projected as samples located in the above row.
当使用Intra_Angular模式时，每个TB都是使用空域上相邻的已重建的像素来预测；
如图6中所示,对于一个尺寸为NxN的TB, 它一共有4N+1个相邻像素用来预测；
在HEVC中，当先前TB解码操作有效时，左下角相邻TB，左相邻TB，上和右上相邻TB的
相邻边界像素均可用作预测；
Intra_Angular模式的预测处理可以从给定的方向的像素位置中推断得到；

为了去掉在参考行和列buffer中的逐像素的切换，对于Intra_Angular[k]中，
k为2-17时，位于上行的像素被投影成位于左列的额外像素；
k为18-34时，位于左列的像素被投影成位于上行的像素；
(也没太搞明白什么意思，到研究标准文档时再细看吧！)


To improve the intrapicture prediction accuracy, the projected
reference sample location is computed with 1/32 sample
accuracy. Bilinear interpolation is used to obtain the value
of the projected reference sample using two closest reference
samples located at integer positions.
为了提高帧内预测的精度，被投影的参考像素精度位置使用1/32像素精度；
为了获得被投影的参考像素，需要对位于整数位置间的两个最近的参考像素使用线性插值；

The prediction process of the Intra_Angular modes is consistent
across all block sizes and prediction directions, whereas
H.264/MPEG-4 AVC uses different methods for its supported
block sizes of 4×4, 8×8, and 16×16. This design consistency
is especially desirable since HEVC supports a greater variety
of TB sizes and a significantly increased number of prediction
directions compared to H.264/MPEG-4 AVC.
Intra_Angular模式的预测处理可以用于所有块尺寸和预测方向；
这一点是不同于H.264/MPEG-4 AVC对不同的块尺寸(4x4,8x8,16x16)使用不同的帧内预测模式集；
HEVC的这个设计很适合大的TB尺寸预测；

3) 帧内planar和帧内DC预测
Intra_Planar and Intra_DC Prediction: 

In addition to Intra_Angular prediction that targets regions with strong
directional edges, HEVC supports two alternative prediction
methods, Intra_Planar and Intra_DC, for which similar modes
were specified in H.264/MPEG-4 AVC. While Intra_DC prediction
uses an average value of reference samples for the
prediction, average values of two linear predictions using four
corner reference samples are used in Intra_Planar prediction
to prevent discontinuities along the block boundaries. The
Intra_Planar prediction mode is supported at all block sizes in
HEVC, while H.264/MPEG-4 AVC supports plane prediction
only when the luma PB size is 16×16, and its plane prediction
operates somewhat differently from the planar prediction in HEVC.
除了Intra_Angular预测外，HEVC还和H.264/MPEG-4 AVC一样，支持Intra_Planar, Intra_DC预测模式；
. Intra_DC     使用参考像素的均值进行预测；
. Intra_Planar 使用四个角的参考像素得到的两个线性预测的均值; 

在HEVC中，Intra_Planar预测模式可用于所有块尺寸，
不像在H.264/MPEG-4 AVC中，plane预测模式只能用在16x16的亮度块，且它们的方法也不一样；

4) 参考像素平滑
Reference Sample Smoothing: 

In HEVC, the reference
samples used for the intrapicture prediction are sometimes
filtered by a three-tap [1 2 1]/4 smoothing filter in a manner
similar to what was used for 8×8 intrapicture prediction in
H.264/MPEG-4 AVC. HEVC applies smoothing operations
more adaptively, according to the directionality, the amount of
detected discontinuity, and the block size. As in H.264/MPEG-
4 AVC, the smoothing filter is not applied for 4×4 blocks. For
8×8 blocks, only the diagonal directions, Intra_Angular[k]
with k = 2, 18, or 34, use the reference sample smoothing.
在HEVC中，用于帧内预测的参考像素可以使用三阶([1 2 1]/4)平滑滤波器来平滑，
这个和H.264/MPEG-4 AVC中的8x8帧内预测模式一样；
HEVC可以根据预测方向，不连续的数量，和块大小对平滑操作灵活使用；
而在H.264/MPEG-4 AVC中，平滑操作不能用于4x4块，对于8x8块，也只能用于对角线预测方向；
HEVC则可以对Intra_Angular[k], k=2, 18, 34，使用参考像素平滑；

For 16×16 blocks, the reference samples are filtered for
most directions except the near-horizontal and near-vertical
directions, k in the range of 9–11 and 25–27. For 32×32
blocks, all directions except the exactly horizontal (k = 10)
and exactly vertical (k = 26) directions use the smoothing
filter, and when the amount of detected discontinuity exceeds a
threshold, bilinear interpolation from three neighboring region
samples is applied to form a smooth prediction.
对于16x16块，除了近水平和近垂直方向外，可以对k为9-11和25-27预测方向使用参考像素平滑滤波；
对于32x32块，除了水平(k=10)和垂直(k=26)方向外，可以对所有剩下的预测方向使用参考像素平滑滤波；
当不连续侦测的数量超过阈值时，可以对三个相邻区域像素使用线性插值来做平滑预测；

The Intra_Planar mode also uses the smoothing filter when
the block size is greater than or equal to 8×8, and the
smoothing is not used (or useful) for the Intra_DC case.
当块尺寸等于或大于8x8时，可以Intra_Planar模式使用平滑滤波；
不能对Intra_DC模式使用平滑滤波操作，因为本来对这个模式也没有作用；

5) 边界值平滑
Boundary Value Smoothing: 

To remove discontinuities along block boundaries, in three modes, Intra_DC (mode 1)
and Intra_Angular[k] with k = 10 or 26 (exactly horizontal
or exactly vertical), the boundary samples inside the TB
are replaced by filtered values when the TB size is smaller
than 32 × 32. For Intra_DC mode, both the first row and
column of samples in the TB are replaced by the output of
a two-tap [3 1]/4 filter fed by their original value and the
adjacent reference sample. In horizontal (Intra_Angular[10])
prediction, the boundary samples of the first column of the
TB are modified such that half of the difference between
their neighbored reference sample and the top-left reference
sample is added. This makes the prediction signal more
smooth when large variations in the vertical direction are
present. In vertical (Intra_Angular[26]) prediction, the same
is applied to the first row of samples.
为了去掉块边界的不连续性，对于这三种模式，
Intra_DC(mode 1),
Intra_Angular[k], k = 10, 26,
当TB的尺寸小于32x32时，TB内的边界像素将被滤波后的值替代；

对于Intra_DC模式，TB内的第一行和第一列的像素，
    使用以原像素和相邻像素作为二阶([3 1]/4)滤波器输入的输出值代替；
对于Intra_Angular[10], 水平方向，TB的第一列边界像素，
    使用相邻参考像素的差值，再加上左上角参考像素除二来代替；
对于Intra_Angular[26], 垂直方向，TB的第一行边界像素， 
    和水平的处理类似；
(可能理解不是很准确:-)

6) 参考像素替换
Reference Sample Substitution: 

The neighboring reference samples are not available at the slice or tile
boundaries. In addition, when a loss-resilience feature known
as constrained intra prediction is enabled, the neighboring
reference samples inside any interpicture-predicted PB
are also considered not available in order to avoid letting
potentially corrupted prior decoded picture data propagate
errors into the prediction signal. While only Intra_DC
prediction mode is allowed for such cases in H.264/MPEG-4
AVC, HEVC allows the use of other intrapicture prediction
modes after substituting the nonavailable reference sample
values with the neighboring available reference sample values.
在片和瓦片的边界，相邻参考像素是无效的；
另外，当受限帧内预测的丢失恢复功能开启后，为了避免由前面的解码图像造成差错扩散，
所有帧间预测PB的相邻参考像素也是无效的；
在H.264/MPEG-4 AVC中只有Intra_DC预测模式能使用这个功能；
HEVC则允许使用相邻有效的参考像素值替换无效的参考像素值后用于其它帧内预测模式；

7) 模式编码
Mode Coding: 

HEVC supports a total of 33 Intra_Angular prediction modes and Intra_Planar and
Intra_DC prediction modes for luma prediction for all block
sizes. Due to the increased number of directions, HEVC
considers three most probable modes (MPMs) when coding the
luma intrapicture prediction mode predictively, rather than the
one most probable mode considered in H.264/MPEG-4 AVC.
Among the three most probable modes, the first two are initialized
by the luma intrapicture prediction modes of the above
and left PBs if those PBs are available and are coded using an
intrapicture prediction mode. Any unavailable prediction mode
is considered to be Intra_DC. The PB above the luma CTB is
always considered to be unavailable in order to avoid the need
to store a line buffer of neighboring luma prediction modes.
HEVC对所有块尺寸的亮度预测支持33种Intra_Angular，Intra_Planar, Intra_DC预测模式；
由于方向的增加，HEVC提供对亮度帧内预测模式的三种最可能模式(MPM)的预测；
不同于H.264/MPEG-4 AVC只提供最可能模式的预测；
在这三个最可能模式中，如果相邻的上方和左方的PB都有效且是帧内预测编码模式，
则前两个使用上方和左方的PB的亮度帧内预测模式来初始化；
所有无效的预测模式都被认为是Intra_DC；
为了避免需要存储相邻亮度预测模式的一行buffer，亮度CTB之上的PB都认为是无效的；

When the first two most probable modes are not equal,
the third most probable mode is set equal to Intra_Planar,
Intra_DC, or Intra_Angular[26] (vertical), according to which
of these modes, in this order, is not a duplicate of one of
the first two modes. When the first two most probable modes
are the same, if this first mode has the value Intra_Planar
or Intra_DC, the second and third most probable modes are
assigned as Intra_Planar, Intra_DC, or Intra_Angular[26],
according to which of these modes, in this order, are not
duplicates. When the first two most probable modes are the
same and the first mode has an Intra_Angular value, the
second and third most probable modes are chosen as the two
angular prediction modes that are closest to the angle (i.e., the
value of k) of the first.
当前两个最可能模式不相等时，
  则依据Intra_Planar，Intra_DC, 或Intra_Angular[26](垂直)这三种模式，
  将第三个最可能预测模式设为在顺序上不和前两种模式重复的模式；
当前两个最可能模式是相等时，
  且第一个模式的值为Intra_Planar或Intra_DC,
  则依据Intra_Planar，Intra_DC, 或Intra_Angular[26](垂直)这三种模式，
  将第二个和第三个模式依据顺序上的不重复设为这三个模式中的两个；
当前两个最可能模式是相等时，
  且第一个模式的值为Intra_Angular，
  则将第二个和第三个最可能模式设为和第一个模式在方向最近的预测方向值；

In the case that the current luma prediction mode is one
of three MPMs, only the MPM index is transmitted to the
decoder. Otherwise, the index of the current luma prediction
mode excluding the three MPMs is transmitted to the decoder
by using a 5-b fixed length code.
如果当前的亮度预测模式是三个最可能模式(MPM)中的一种，
  则只有MPM的索引号被传输给解码器；
否则，
  除了三个最可能模式外，当前亮度预测模式的索引也要使用5比特的定长码字传输给解码器；

For chroma intrapicture prediction, HEVC allows
the encoder to select one of five modes: Intra_Planar,
Intra_Angular[26] (vertical), Intra_Angular[10] (horizontal),
Intra_DC, and Intra_Derived. The Intra_Derived mode
specifies that the chroma prediction uses the same angular
direction as the luma prediction. With this scheme, all angular
modes specified for luma in HEVC can, in principle, also be
used in the chroma prediction, and a good tradeoff is achieved
between prediction accuracy and the signaling overhead. The
selected chroma prediction mode is coded directly (without
using an MPM prediction mechanism).
对于色度帧内预测模式，HEVC允许其选择下面五种模式：
 Intra_Planar,
 Intra_Angular[26](垂直),
 Intra_Angular[10](水平),
 Intra_DC,
 Intra_Derived,
中的一种,
其中，Intra_Derived是将色度预测模式设定为对应亮度块的帧内方向预测模式；
对于这样一种设计， 在原则上，HEVC支持的所有亮度方向预测模式在色度上也同样支持；
这样的设计很好地取得了预测精度和信号传输开销间的平衡；
被选择的色度预测模式是直接编码的(不使用对预测模式进行预测的机制);

六二，直方大，不习无不利。
【白话】六二，正直、端正、广大，具备这样的品质，即使不学习也不会有什么不利。


《象》曰：六二之动，直以方也，“不习无不利”，地道光也。
【白话】《象辞》说：六二爻若是出现变化的话，总是表现出正直、端正的性质。
“即使不学习也不会有什么不利”，是因为地德广大，包容万物的缘故。

​