H.266/VVC代码学习:MIP技术相关代码之initIntraMip函数

initIntraMip函数主要是对参考像素进行下采样并为MIP矩阵乘法准备输入数据，函数结构如下：

MIP根据块尺寸可以分为以下三种情况：

	块尺寸	下采样后的边界长度 m_reducedBdrySize	矩阵乘法输出边界长度 m_reducedPredSize
mipSizeId = 0	4x4	2	4
mipSizeId = 1	4xN、Nx4、8x8	4	4
mipSizeId = 2	其余块	4	8

initIntraMip函数主要是准备边界参考像素，并调用prepareInputForPred函数为MIP预测准备输入数据

注意：MIP使用的参考像素是未经过滤波的参考像素

initIntraMip函数代码如下所示：

void IntraPrediction::initIntraMip( const PredictionUnit &pu, const CompArea &area )
{
  CHECK( area.width > MIP_MAX_WIDTH || area.height > MIP_MAX_HEIGHT, "Error: block size not supported for MIP" );

  // prepare input (boundary) data for prediction
  // 准备输入（边界）数据进行预测
  // MIP使用未滤波的参考像素
  CHECK( m_ipaParam.refFilterFlag, "ERROR: unfiltered refs expected for MIP" );
#if JVET_R0350_MIP_CHROMA_444_SINGLETREE
  Pel       *ptrSrc     = getPredictorPtr(area.compID);//获取参考像素
  const int  srcStride  = m_refBufferStride[area.compID];
  const int  srcHStride = 2;

  m_matrixIntraPred.prepareInputForPred(CPelBuf(ptrSrc, srcStride, srcHStride), area,
                                        pu.cu->slice->getSPS()->getBitDepth(toChannelType(area.compID)), area.compID);
#else
  Pel *ptrSrc = getPredictorPtr( COMPONENT_Y );
  const int srcStride  = m_refBufferStride[COMPONENT_Y];
  const int srcHStride = 2;

  m_matrixIntraPred.prepareInputForPred( CPelBuf( ptrSrc, srcStride, srcHStride ), area, pu.cu->slice->getSPS()->getBitDepth( CHANNEL_TYPE_LUMA ) );
#endif
}

prepareInputForPred函数主要分为以下四个步骤：

• Step 1: 保存块大小并计算MIP相关参数，通过调用initPredBlockParams函数实现
• Step 2: 获取输入数据（上一行参考像素和左一列参考像素）
• Step 3: 通过Haar下采样计算缩减边界，通过boundaryDownsampling1D函数实现
• Step 4: 推导矩阵乘法输入向量

prepareInputForPred函数代码如下所示

#if JVET_R0350_MIP_CHROMA_444_SINGLETREE
void MatrixIntraPrediction::prepareInputForPred(const CPelBuf &pSrc, const Area &block, const int bitDepth,
                                                const ComponentID compId)
{
  m_component = compId;
#else
void MatrixIntraPrediction::prepareInputForPred(const CPelBuf &pSrc, const Area& block, const int bitDepth)
{
#endif
  // Step 1: Save block size and calculate dependent values
  // Step 1: 保存块大小并计算MIP相关参数
  initPredBlockParams(block);

  // Step 2: Get the input data (left and top reference samples)
  // Step 2: 获取输入数据（左上参考像素）
  // 获取上一行参考像素
  m_refSamplesTop.resize(block.width);
  for (int x = 0; x < block.width; x++)
  {
    m_refSamplesTop[x] = pSrc.at(x + 1, 0);
  }
  // 获取左一列参考像素
  m_refSamplesLeft.resize(block.height);
  for (int y = 0; y < block.height; y++)
  {
    m_refSamplesLeft[y] = pSrc.at(y + 1, 1);
  }

  // Step 3: Compute the reduced boundary via Haar-downsampling (input for the prediction)
  // Step 3: 通过Haar下采样计算缩减边界（预测输入）
  // 下采样后输入向量的尺寸为4或者8
  const int inputSize = 2 * m_reducedBdrySize;
  // 不需要转置时，下采样像素的顺序：先上后左
  m_reducedBoundary          .resize( inputSize );
  // 转置时，下采样像素的顺序：先左后上
  m_reducedBoundaryTransposed.resize( inputSize );

  int* const topReduced = m_reducedBoundary.data();
  boundaryDownsampling1D( topReduced, m_refSamplesTop.data(), block.width, m_reducedBdrySize );

  int* const leftReduced = m_reducedBoundary.data() + m_reducedBdrySize;
  boundaryDownsampling1D( leftReduced, m_refSamplesLeft.data(), block.height, m_reducedBdrySize );

  int* const leftReducedTransposed = m_reducedBoundaryTransposed.data();
  int* const topReducedTransposed  = m_reducedBoundaryTransposed.data() + m_reducedBdrySize;
  for( int x = 0; x < m_reducedBdrySize; x++ )
  {
    topReducedTransposed[x] = topReduced[x];
  }
  for( int y = 0; y < m_reducedBdrySize; y++ )
  {
    leftReducedTransposed[y] = leftReduced[y];
  }

  // Step 4: Rebase the reduced boundary
  // Step 4: 缩小边界
  // 推导矩阵乘法输入向量p，mipSizeId=0/1和mipSizeId=2的推导方法不一样
  m_inputOffset       = m_reducedBoundary[0];
  m_inputOffsetTransp = m_reducedBoundaryTransposed[0];
  const bool hasFirstCol = (m_sizeId < 2);

  m_reducedBoundary          [0] = hasFirstCol ? ((1 << (bitDepth - 1)) - m_inputOffset      ) : 0; // first column of matrix not needed for large blocks
  m_reducedBoundaryTransposed[0] = hasFirstCol ? ((1 << (bitDepth - 1)) - m_inputOffsetTransp) : 0;
  for (int i = 1; i < inputSize; i++)
  {
    m_reducedBoundary          [i] -= m_inputOffset;
    m_reducedBoundaryTransposed[i] -= m_inputOffsetTransp;
  }
}

1、初始化MIP相关参数

initPredBlockParams函数是用来初始化MIP相关参数，主要是根据当前块的尺寸来初始化mipSizeId，然后根据mipSizeId初始化下采样后的边界长度、矩阵乘法输出边界长度和上采样因子

void MatrixIntraPrediction::initPredBlockParams(const Size& block)
{
  //获得当前块尺寸
  m_blockSize = block;
  // init size index
  // 根据当前块尺寸初始化sizeId
  m_sizeId = getMipSizeId( m_blockSize );
  // init reduced boundary size
  // 初始缩减边界尺寸
  // 对于4x4的块宽度和高度分别缩减为2个像素
  // 对于其余尺寸的块宽度和高度分别缩减为4个像素
  m_reducedBdrySize = (m_sizeId == 0) ? 2 : 4;
  // init reduced prediction size
  // 初始化缩减预测后的尺寸
  // 对于mipSizeId = 0、1的块，MIP预测后输出4x4的块
  // 对于mipSizeId = 2的块，MIP预测后输出8x8的块
  m_reducedPredSize = ( m_sizeId < 2 ) ? 4 : 8;
  // init upsampling factors
  // 初始上采样因子
  m_upsmpFactorHor = m_blockSize.width  / m_reducedPredSize;
  m_upsmpFactorVer = m_blockSize.height / m_reducedPredSize;

  CHECKD( (m_upsmpFactorHor < 1) || ((m_upsmpFactorHor & (m_upsmpFactorHor - 1)) != 0), "Need power of two horizontal upsampling factor." );
  CHECKD( (m_upsmpFactorVer < 1) || ((m_upsmpFactorVer & (m_upsmpFactorVer - 1)) != 0), "Need power of two vertical upsampling factor." );
}

2、下采样

边界参考像素的下采样过程是由boundaryDownsampling1D函数实现的，下采样过程其实就是对边界参考像素求平均的过程，以8x8的块为例，如下图所示，上一行存在8个参考像素，通过对两两相邻的参考像素求平均后获得4个下采样后的参考像素，左一列参考像素地下采样过程同理。

/*
一维下采样
reducedDst表示下采样后的边界
fullSrc表示下采样前的边界
srcLen表示下采样前的边界长度
dstLen表示下采样后的边界长度
*/
void MatrixIntraPrediction::boundaryDownsampling1D(int* reducedDst, const int* const fullSrc, const SizeType srcLen, const SizeType dstLen)
{
  if (dstLen < srcLen)
  {
    //当下采样后的边界尺寸小于当前块的边界尺寸时，需要进行下采样，下采样操作即相当于求平均操作
    // Create reduced boundary by downsampling 通过下采样创建缩小边界
    const SizeType downsmpFactor = srcLen / dstLen;
    const int log2DownsmpFactor = floorLog2(downsmpFactor);
    const int roundingOffset = (1 << (log2DownsmpFactor - 1));

    SizeType srcIdx = 0;
    for( SizeType dstIdx = 0; dstIdx < dstLen; dstIdx++ )
    {
      int sum = 0;
      for( int k = 0; k < downsmpFactor; k++ )
      {
        sum += fullSrc[srcIdx++];
      }
      reducedDst[dstIdx] = (sum + roundingOffset) >> log2DownsmpFactor;
    }
  }
  else
  {
    // Copy boundary if no downsampling is needed 如果不需要下采样，则复制边界
    for (SizeType i = 0; i < dstLen; ++i)
    {
      reducedDst[i] = fullSrc[i];
    }
  }
}

下采样过程结束后，根据mipTransposeFlag标志将下采样后的上参考像素和左参考像素排列成向量pTemp，排列方式如下：

• mipTransposeFlag = 0时，pTemp = [redTop,redLeft]
• mipTransposeFlag = 1时，pTemp = [redLeft,redTop]

3、推导矩阵乘法输入向量

矩阵乘法输入向量的推导方法和mipSizeId有关，输入向量p的构造过程如下所示，其中inSize = 2*m_reducedBdrySize