H.266/VVC代码学习笔记8：VTM5.0中帧内亮度预测函数——estIntraPredLumaQT()函数

今天对VTM5.0中的亮度帧内预测的入口函数estIntraPredLumaQT()进行详细的学习

estIntraPredLumaQT中完成了亮度分量的帧内预测，其主要流程如下：
一、初始化各种参数。
二、为了减少最终RDcost的次数，降低编码端的复杂度，VVC的VTM5.0版本默认使用帧内快速搜索算法，主要经过两轮的SATD的粗选阶段和一轮RDcost的细选阶段
 1、第一轮的SATD粗选，首先对35种非扩展的传统的亮度帧内模式进行第一轮SATD循环粗选，这里粗选的时候使用predIntraAng函数计算预测值，然后计算比较哈达玛失真，取SAD和SATD（HAD）之中的最小值作为最小的代价，然后调用updateCandList函数构建全率失真优化候选列表。全率失真优化候选列表长度numModesForFullRD由块宽度决定。
 2、第二轮的SATD粗选，如今的VTM5.0因为引入了很多的新模式，因此整个第二轮的流程复杂了很多，如下：
   ①对第一轮后剩余的扩展角度模式的SATD粗选，这里粗选的时候仍然用使用predIntraAng函数计算预测值，然后计算比较哈达玛失真，取SAD和SATD（HAD）之中的最小值作为最小的代价，然后调用updateCandList函数构建全率失真优化候选列表。全率失真优化候选列表长度numModesForFullRD由块宽度决定。

   ②对传统的MPM列表进行SATD的粗选，选出最有可能的几种模式加入到RD候选列表中去

   ③在两轮传统角度模式的SATD以及MPM的SATD结束以后，对使用哈达吗的MIp模式单独进
行候选模式的推导，关于Mip模式的技术细节我之前的博客已经讲过啦，这里就不做过多阐释，直接给出链接H.266/VVC代码学习笔记4：带你深入解析VTM5.0中的最新技术ALWIP，在Mip的SATD之后这里对RD候选列表CandHadList进行第一次缩减reduceHadCandList，也是为了再次降低编码端的复杂度。

   ④FastUDI最可能的MPM列表模式，这里设置了一个numCand，定义MPM列表中最可能的模式数量,然后只对MPM中最可能的那几个模式循环进行SATD粗选，其余模式不进行，在一定程度上也节省了几次SATD的比较和计算，然后更新RD候选列表CandHadList，这里对于ISP模式的情况下还单独进行了处理，在当前CU是ISP模式的时候，将不加入多参考情况的ISP模式添加到水平/垂直RD候选列表中，需要定义一个独立的定义一个指示器列表listPointer，存入非MRL的ISP模式。对每一个ISP的分区独立定义一个MPM列表和最可能的MPM候选模式的数量numCandISP，对每个候选的MPM模式循环遍历进行STAD的粗选，如果遍历到的模式不包括在最可能模式中，将ISP的最可能模式push到listPointer中。

   ⑤独立的对Mip模式定义一个3MPM列表，对该Mip的MPM列表中的每一个模式循环遍历SATD粗选，将最优的模式加入到RD候选列表CandHadList。关于Mip模式的MPM列表的构造具体的技术细节我在之前的博客已经讲过了。这里直接给出链接：H.266/VVC代码学习笔记7：VTM5.0中Mip模式的MPM列表构造函数——getMipMPMs()

   ⑥重置RD候选模式列表的数量，然后进行第二次RD候选列表CandHadList的缩减，为了排除可能性最低的那些模式，降低编码端RDcost的复杂度。

   ⑦如果当前的CU是ISP模式，从ISP专属的列表listPointer中移除非MPM模式，对每一个子分区的ISP列表进行循环重置操作。这一步操作之后，先不要去使用numModesForFullRD作为最终的RD候选列表的大小。

   ⑧如果当前的CU是ISP模式，则创造一个完整的RD候选列表，其中包括使用常规intra、mrl和isp的所有内部模式。并且使用使用快速ISP模式，这里定义了一个indexFirstMode变量，用来找到RD候选列表CandHadList中的第一个非MRL和非Mip模式。然后对RD候选模式列表中的每个模式进行循环遍历处理，对RD候选列表CandHadList中的模式进行排序操作，具体操作步骤如下：
     （1）将indexFirstMode指示的模式移到候选列表的开头；
     （2）然后在第一个非MRL模式之后插入所有的ISP模式
     （3）在当前的列表的结尾插入所有的ISP模式

在经过了两轮非常长的SATD的粗选过程终于到了最后一步的RDcost的细选阶段：
 3、第三轮的RDcost细选，check 所有的RD候选模式（使用RDcost）
   ①首先重新定义一个临时的RD候选列表uiRdModeListTemp，先把非ISP且非MIP的模式放到重新定义的RD候选列表的前面，再把ISP或者MIP的模式放到重新定义的RD候选列表的后面，最后将临时的RD候选列表uiRdModeListTemp中的模式按顺序放入原来的RD候选模式列表uiRdModeList中，这一步的目的就是为了给RD候选列表重新排序。

   ②在确保numModesForFullRD和RD候选列表数目相等的条件下，才进入到最终对于RD候选列表中的模式的进一步精选RDcost操作，这里有一个大循环，对每个模式进行RDcost的遍历，这里调用xRecurIntraCodingLumaQT函数计算预测值以及残差值，然后对残差值进行变换量化并且计算重建值，最后计算率失真代价，注意在这里该函数倒数第三个参数是mtsFirstCheckId=true，表示会按照四叉树的方式继续向下划分。然后比较率失真代价，check每个模式的RD cost，逐次筛选出cost最小的模式，更新最优模式。最终遍历结束，选出最优模式。

   ③最后收尾工作，判断cost最优的模式是否是Mip模式以及是否使用多参考行，并将选出的最优的模式传给intraDir，重置上下文模型。

下面是整个函数的所有代码流程，我都加了详细的注释，方便大家参考和阅读,这里我建议大家把代码放到VS中去看，因为里面有一些宏注释的内容，在这里看容易混淆，但在VS里就很清楚：

bool IntraSearch::estIntraPredLumaQT( CodingUnit &cu, Partitioner &partitioner, const double bestCostSoFar, bool mtsCheckRangeFlag, int mtsFirstCheckId, int mtsLastCheckId, bool moreProbMTSIdxFirst )

{
  CodingStructure       &cs            = *cu.cs;//获取当前CU的编码结构
  const SPS             &sps           = *cs.sps;//获取该编码结构的语法元素
  const uint32_t             uiWidthBit    = g_aucLog2[partitioner.currArea().lwidth() ];
  const uint32_t             uiHeightBit   =                   g_aucLog2[partitioner.currArea().lheight()];

  // Lambda calculation at equivalent Qp of 4 is recommended because at that Qp, the quantization divisor is 1.
  //建议在等效qp为4时进行lambda的计算，因为在该qp时，用于量化的除数为1。
  const double sqrtLambdaForFirstPass = m_pcRdCost->getMotionLambda(cu.transQuantBypass) / double(1 << SCALE_BITS);


  //===== loop over partitions =====
  //循环分区


  //上下玩模型的开始
  const TempCtx ctxStart          ( m_CtxCache, m_CABACEstimator->getCtx() );
#if JVET_N0217_MATRIX_INTRAPRED

  //Mip模式的flag上下文模型
  const TempCtx ctxStartMipFlag    ( m_CtxCache, SubCtx( Ctx::MipFlag,          m_CABACEstimator->getCtx() ) );
 
  //Mip模式的上下文模型
  const TempCtx ctxStartMipMode    ( m_CtxCache, SubCtx( Ctx::MipMode,          m_CABACEstimator->getCtx() ) );
#endif
  //TSP模式的上下文模型
  const TempCtx ctxStartIspMode    ( m_CtxCache, SubCtx( Ctx::ISPMode,          m_CABACEstimator->getCtx() ) );
#if JVET_N0185_UNIFIED_MPM
  //Planar模式的flag上下文模型
  const TempCtx ctxStartPlanarFlag ( m_CtxCache, SubCtx( Ctx::IntraLumaPlanarFlag, m_CABACEstimator->getCtx() ) );
#endif
  //其余帧内模式的上下文模型
  const TempCtx ctxStartIntraMode(m_CtxCache, SubCtx(Ctx::IntraLumaMpmFlag, m_CABACEstimator->getCtx()));


  //多参考行的索引
  const TempCtx ctxStartMrlIdx      ( m_CtxCache, SubCtx( Ctx::MultiRefLineIdx,        m_CABACEstimator->getCtx() ) );
//检查如果CU没有对应的PU块的则抛出异常
  CHECK( !cu.firstPU, "CU has no PUs" );
  const bool keepResi   = cs.pps->getPpsRangeExtension().getCrossComponentPredictionEnabledFlag() || KEEP_PRED_AND_RESI_SIGNALS;

#if JVET_N0193_LFNST
  // variables for saving fast intra modes scan results across multiple LFNST passes
  bool LFNSTLoadFlag = sps.getUseLFNST() && cu.lfnstIdx != 0;
  bool LFNSTSaveFlag = sps.getUseLFNST() && cu.lfnstIdx == 0;

  LFNSTSaveFlag &= sps.getUseIntraMTS() ? cu.mtsFlag == 0 : true;

  const uint32_t lfnstIdx = cu.lfnstIdx;
#endif


#if JVET_N0193_LFNST

  //当前亮度CU的宽和高
  const int width  = partitioner.currArea().lwidth();
  const int height = partitioner.currArea().lheight();

  // Marking MTS usage for faster MTS
  // 0: MTS is either not applicable for current CU (cuWidth > MTS_INTRA_MAX_CU_SIZE or cuHeight > MTS_INTRA_MAX_CU_SIZE), not active in the config file or the fast decision algorithm is not used in this case
  // 1: MTS fast algorithm can be applied for the current CU, and the DCT2 is being checked
  // 2: MTS is being checked for current CU. Stored results of DCT2 can be utilized for speedup
  uint8_t mtsUsageFlag = 0;
  const int maxSizeEMT = MTS_INTRA_MAX_CU_SIZE;
  if( width <= maxSizeEMT && height <= maxSizeEMT && sps.getUseIntraMTS() )
  {
    mtsUsageFlag = ( sps.getUseLFNST() && cu.mtsFlag == 1 ) ? 2 : 1;
  }

  if( width * height < 64 && !m_pcEncCfg->getUseFastLFNST() )
  {
    mtsUsageFlag = 0;
  }
#endif
#if JVET_N0193_LFNST
#if INCLUDE_ISP_CFG_FLAG
  int nOptionsForISP = ( sps.getUseISP() && cu.mtsFlag == 0 && cu.lfnstIdx == 0 ) ? NUM_INTRA_SUBPARTITIONS_MODES : 1;
#endif
#endif

  //定义一个当前最优的代价
  double bestCurrentCost = bestCostSoFar;

  int ispOptions[NUM_INTRA_SUBPARTITIONS_MODES] = { 0 };
  if( nOptionsForISP > 1 )
  {
#if MAX_TB_SIZE_SIGNALLING
#else

    //ISP模式的划分
    auto splitsThatCanBeUsedForISP = CU::canUseISPSplit( width, height, MAX_TB_SIZEY );
#endif
    if( splitsThatCanBeUsedForISP == CAN_USE_VER_AND_HORL_SPLITS )
    {
      const CodingUnit* cuLeft  = cu.ispMode != NOT_INTRA_SUBPARTITIONS ? cs.getCU( cs.area.blocks[partitioner.chType].pos().offset( -1, 0 ), partitioner.chType ) : nullptr;
      const CodingUnit* cuAbove = cu.ispMode != NOT_INTRA_SUBPARTITIONS ? cs.getCU( cs.area.blocks[partitioner.chType].pos().offset( 0, -1 ), partitioner.chType ) : nullptr;
      bool ispHorIsFirstTest = CU::firstTestISPHorSplit( width, height, COMPONENT_Y, cuLeft, cuAbove );
      if( ispHorIsFirstTest )
      {
        ispOptions[1] = HOR_INTRA_SUBPARTITIONS;
        ispOptions[2] = VER_INTRA_SUBPARTITIONS;
      }
      else
      {
        ispOptions[1] = VER_INTRA_SUBPARTITIONS;
        ispOptions[2] = HOR_INTRA_SUBPARTITIONS;
      }
    }
    else if( splitsThatCanBeUsedForISP == HOR_INTRA_SUBPARTITIONS )
    {
      nOptionsForISP = 2;
      ispOptions[1] = HOR_INTRA_SUBPARTITIONS;
    }
    else if( splitsThatCanBeUsedForISP == VER_INTRA_SUBPARTITIONS )
    {
      nOptionsForISP = 2;
      ispOptions[1] = VER_INTRA_SUBPARTITIONS;
    }
    else
    {
      nOptionsForISP = 1;
    }
  }
  if( nOptionsForISP > 1 )
  {
    //variables for the full RD list without MRL modes
    m_rdModeListWithoutMrl      .clear();
    m_rdModeListWithoutMrlHor   .clear();
    m_rdModeListWithoutMrlVer   .clear();
    //variables with data from regular intra used to skip ISP splits
    m_intraModeDiagRatio        .clear();
    m_intraModeHorVerRatio      .clear();
    m_intraModeTestedNormalIntra.clear();
  }

#if JVET_N0413_RDPCM
#if JVET_N0193_LFNST
  const bool testBDPCM = m_pcEncCfg->getRDPCM() && CU::bdpcmAllowed( cu, ComponentID( partitioner.chType ) ) && cu.mtsFlag == 0 && cu.lfnstIdx == 0;
#else
#endif
#endif
#if JVET_N0217_MATRIX_INTRAPRED
  //哈达吗变换（即SATD）粗选后的候选列表
  static_vector uiHadModeList;
#else
#endif
  //定义代价候选列表
  static_vector CandCostList;

  //定义哈达吗变换（SATD/HAD）后的候选代价列表
  static_vector CandHadList;


  auto &pu = *cu.firstPU;
#if JVET_N0193_LFNST
  bool validReturn = false;
#endif
  {
    CandHadList.clear();
    CandCostList.clear();
    uiHadModeList.clear();
#if !JVET_N0217_MATRIX_INTRAPRED
    extendRefList.clear();
#endif

    CHECK(pu.cu != &cu, "PU is not contained in the CU");

    //===== determine set of modes to be tested (using prediction signal only) =====
    /******************** 确定要测试的模式集和（仅使用预测信号）**************************/
    int numModesAvailable = NUM_LUMA_MODE; // 所有67种传统帧内模式的数量，不包括MIp模式，MIp模式另行处理
#if JVET_N0217_MATRIX_INTRAPRED
    
    //加快速编码版本的MIp模式
    const bool fastMip    = sps.getUseMIP() && m_pcEncCfg->getUseFastMIP();
#if JVET_N0193_LFNST
    //Mip模式被允许
    const bool mipAllowed = sps.getUseMIP() && ( cu.lfnstIdx == 0 ) && isLuma( partitioner.chType ) && pu.lwidth() <= MIP_MAX_WIDTH && pu.lheight() <= MIP_MAX_HEIGHT;
#endif

    //是否是Mip模式的测试
    const bool testMip    = mipAllowed && mipModesAvailable( pu.Y() ) && !(fastMip && (cu.lwidth() > 2 * cu.lheight() || cu.lheight() > 2 * cu.lwidth())); 
    
    
    //定义一个不使用哈达吗变换（即SAD）的RD候选列表
    static_vector uiRdModeList;

#endif
    /*这里定义的就是能够进入到最终的RDcost环节的最优的几种模式的数量，numModesForFullRD初始化为3*/
    int numModesForFullRD = 3;
    numModesForFullRD = g_aucIntraModeNumFast_UseMPM_2D[uiWidthBit - MIN_CU_LOG2][uiHeightBit - MIN_CU_LOG2];

#if JVET_N0193_LFNST
    if( mtsUsageFlag != 2 )
#endif
    {
      // this should always be true
      //如果当前PU无效就抛出异常
      CHECK( !pu.Y().valid(), "PU is not valid" );
#if ENABLE_JVET_L0283_MRL

      //是否是CTU的第一行
      bool isFirstLineOfCtu = (((pu.block(COMPONENT_Y).y)&((pu.cs->sps)->getMaxCUWidth() - 1)) == 0);

      //使用的扩展参考行的数量，如果为CTU的第一行，则最多只能用1行；如果不是，则最多可以用三行
      int numOfPassesExtendRef = (isFirstLineOfCtu ? 1 : MRL_NUM_REF_LINES);
#endif
      pu.multiRefIdx = 0;

      if( numModesForFullRD != numModesAvailable )//numModesForFullRD的数量应该属少于所有可用的帧内模式的数量的，因此这里是主要的RD的入口
      {

        CHECK( numModesForFullRD >= numModesAvailable, "Too many modes for full RD search" );

        const CompArea &area = pu.Y();//获取当前亮度PU的区域

        PelBuf piOrg         = cs.getOrgBuf(area);//获取当前亮度PU区域的亮度原始值
        PelBuf piPred        = cs.getPredBuf(area);//当前亮度PU区域的亮度预测值，为期分配固定大小的内存空间

#if JVET_N0363_INTRA_COST_MOD
        DistParam distParamSad;//定义一个使用SAD的失真参数


        DistParam distParamHad;//定义一个使用SATD（加了哈达吗变换）的失真参数
#endif

        if (cu.slice->getReshapeInfo().getUseSliceReshaper() && m_pcReshape->getCTUFlag())
        {
          CompArea      tmpArea(COMPONENT_Y, area.chromaFormat, Position(0, 0), area.size());
          PelBuf tmpOrg = m_tmpStorageLCU.getBuf(tmpArea);
          tmpOrg.copyFrom(piOrg);
          tmpOrg.rspSignal(m_pcReshape->getFwdLUT());
#if JVET_N0363_INTRA_COST_MOD

          //设置失真参数
          m_pcRdCost->setDistParam(distParamSad, tmpOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, false); // Use SAD cost
          m_pcRdCost->setDistParam(distParamHad, tmpOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y,  true); // Use HAD (SATD) cost

#endif
        }
        else
#if JVET_N0363_INTRA_COST_MOD
        {
          m_pcRdCost->setDistParam(distParamSad, piOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y, false); // Use SAD cost
          m_pcRdCost->setDistParam(distParamHad, piOrg, piPred, sps.getBitDepth(CHANNEL_TYPE_LUMA), COMPONENT_Y,  true); // Use HAD (SATD) cost
        }

#endif

#if JVET_N0363_INTRA_COST_MOD
        distParamSad.applyWeight = false;
        distParamHad.applyWeight = false;

#endif

#if JVET_N0217_MATRIX_INTRAPRED
        if( testMip)
        {
          numModesForFullRD += fastMip? std::max(2, g_aucLog2[std::min(pu.lwidth(), pu.lheight())] - 1) : numModesForFullRD;
        }

        //定义SATD变换候选的数量
        const int numHadCand = (testMip ? 2 : 1) * 3;//是Mip模式为6，非Mip模式为3

        //*** Derive (regular) candidates using Hadamard
        //使用哈达吗变换的推导角度候选（即使用SATD）

        cu.mipFlag = false;

        //===== init pattern for luma prediction =====
        //初始化亮度预测模式
        initIntraPatternChType(cu, pu.Y(), true);
#endif
        bool bSatdChecked[NUM_INTRA_MODE];//定义一个存放已经被STAD粗选过的帧内模式的bool数组
        memset( bSatdChecked, 0, sizeof( bSatdChecked ) );

#if JVET_N0193_LFNST
        if( !LFNSTLoadFlag )
#endif
        {
          //首先对35种非扩展的传统的亮度帧内模式进行第一轮SATD循环粗选
          //0.1.2.4.6.8................50.............66
          for( int modeIdx = 0; modeIdx < numModesAvailable; modeIdx++ )
          {
            uint32_t       uiMode = modeIdx;
#if JVET_N0363_INTRA_COST_MOD
            Distortion minSadHad = 0;
#endif

            // Skip checking extended Angular modes in the first round of SATD
            //这里明确说要跳过扩展的角度模式的SATD粗选
            if( uiMode > DC_IDX && ( uiMode & 1 ) )
            {
              continue;
            }

            bSatdChecked[uiMode] = true;
            //intraDir中存放的就是当前CU最终选中的预测模式
            pu.intraDir[0] = modeIdx;

            initPredIntraParams(pu, pu.Y(), sps);
            if( useDPCMForFirstPassIntraEstimation( pu, uiMode ) )
            {
              encPredIntraDPCM( COMPONENT_Y, piOrg, piPred, uiMode );
            }
            else
            {
              //传统角度预测模式的函数入口
              predIntraAng( COMPONENT_Y, piPred, pu);
            }
#if JVET_N0363_INTRA_COST_MOD
            // Use the min between SAD and HAD as the cost criterion
            //使用SAD和HAD之间的最小值作为代价标准(HAD其实就是SATD，是加了哈达吗变换后的SAD)
            // SAD is scaled by 2 to align with the scaling of HAD
            //SAD的缩放比例为2，与HAD的缩放比例一致。
            minSadHad += std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif

            // NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.

            //
#if JVET_N0217_MATRIX_INTRAPRED 
            //获取Mip模式flag的上下文模型
            m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
            //获取ISP模式的上下文模型
            m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
            //获取亮度Planar模式的上下文模型
            m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
            //获取亮度MPM列表的上下文模型
            m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
            //获取多参考行的上下文模型
            m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );

            uint64_t fracModeBits = xFracModeBitsIntra(pu, uiMode, CHANNEL_TYPE_LUMA);

#if JVET_N0363_INTRA_COST_MOD
            //计算每种模式的Cost，比较相互之间的Cost进行模式更新
            double cost = ( double ) minSadHad + (double)fracModeBits * sqrtLambdaForFirstPass;

            DTRACE(g_trace_ctx, D_INTRA_COST, "IntraHAD: %u, %llu, %f (%d)\n", minSadHad, fracModeBits, cost, uiMode);
#endif

#if JVET_N0217_MATRIX_INTRAPRED
            //更新SAD之后的RD候选模式列表以及代价列表
            updateCandList( ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, uiMode), cost,          uiRdModeList,  CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD
            //更新传统帧内预测模式的候选模式列表
            //就是更新SATD之后的uiHadModeList、CandHadList这些列表
            updateCandList( ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, uiMode), (double)minSadHad, uiHadModeList, CandHadList,  numHadCand );
#endif
#endif
          }
#if JVET_N0193_LFNST
          if( LFNSTSaveFlag )
          {
            // save found best modes
            //保存发现的最优的模式
            m_uiSavedNumRdModesLFNST   = numModesForFullRD;//第一轮SAD粗选后选中的相对最优模式的数量
            m_uiSavedRdModeListLFNST   = uiRdModeList;//被保存下来的RD模式的列表
            m_dSavedModeCostLFNST      = CandCostList;//被保存下来的候选代价的列表
            // PBINTRA fast
            m_uiSavedHadModeListLFNST  = uiHadModeList;
            m_dSavedHadListLFNST       = CandHadList;
            LFNSTSaveFlag              = false;
          }
#endif
        } // NSSTFlag
#if JVET_N0193_LFNST
        else
        {
          //restore saved modes
          numModesForFullRD = m_uiSavedNumRdModesLFNST;
          uiRdModeList      = m_uiSavedRdModeListLFNST;
          CandCostList      = m_dSavedModeCostLFNST;
          //PBINTRA fast
          uiHadModeList     = m_uiSavedHadModeListLFNST;
          CandHadList       = m_dSavedHadListLFNST;

          LFNSTLoadFlag     = false;
        } // !LFNSTFlag
#endif

#if JVET_N0217_MATRIX_INTRAPRED
        //再次检查uiRdModeList的大小是否和numModesForFullRD相等，不相等则抛出异常
        CHECK( uiRdModeList.size() != numModesForFullRD, "Error: RD mode list size" );

        //将第一轮选出来的最优模式放到父模式中
        static_vector parentCandList = uiRdModeList;

#endif

        // Second round of SATD for extended Angular modes

        //这里开始第二轮对剩下的扩展角度模式的SATD粗选
        for (int modeIdx = 0; modeIdx < numModesForFullRD; modeIdx++)
        {
#if JVET_N0217_MATRIX_INTRAPRED
          //定义父模式，父模式放入的是第一轮选出来的最优的模式列表
          unsigned parentMode = parentCandList[modeIdx].modeId;
#endif
          //从扩展模式开始循环处理
          //模式号是从3-65
          if (parentMode > (DC_IDX + 1) && parentMode < (NUM_LUMA_MODE - 1))
          {
            for (int subModeIdx = -1; subModeIdx <= 1; subModeIdx += 2)
            {
              unsigned mode = parentMode + subModeIdx;//给父模式减一或者加一，正好就是上一轮那些最优候选列表中非扩展角度模式的左右相邻的扩展角度模式


              if (!bSatdChecked[mode])
              {
                pu.intraDir[0] = mode;

                initPredIntraParams(pu, pu.Y(), sps);
                if (useDPCMForFirstPassIntraEstimation(pu, mode))
                {
                  encPredIntraDPCM(COMPONENT_Y, piOrg, piPred, mode);
                }
                else
                {
                  predIntraAng(COMPONENT_Y, piPred, pu );
                }
                
#if JVET_N0363_INTRA_COST_MOD
                // Use the min between SAD and SATD as the cost criterion
                //使用SAD和SATD之间的最小值作为代价标准
                // SAD is scaled by 2 to align with the scaling of HAD
                Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif

                // NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.
#if JVET_N0217_MATRIX_INTRAPRED  
                m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
                m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
                m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
                m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
                m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );

                uint64_t fracModeBits = xFracModeBitsIntra(pu, mode, CHANNEL_TYPE_LUMA);

#if JVET_N0363_INTRA_COST_MOD

                //计算第二轮SATD每一种模式的cost
                double cost = (double) minSadHad + (double) fracModeBits * sqrtLambdaForFirstPass;
#endif

#if JVET_N0217_MATRIX_INTRAPRED
                updateCandList( ModeInfo( false, 0, NOT_INTRA_SUBPARTITIONS, mode ), cost,        uiRdModeList,  CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD
                //更新模式候选列表，将最优的扩展角度模式加到模式列表中
                updateCandList( ModeInfo( false, 0, NOT_INTRA_SUBPARTITIONS, mode ), (double)minSadHad, uiHadModeList, CandHadList,  numHadCand );
#endif
#endif

                bSatdChecked[mode] = true;
              }
            }
          }
        }
        if( nOptionsForISP > 1 )
        {
          //we save the list with no mrl modes to keep only the Hadamard selected modes (no mpms)
#if JVET_N0217_MATRIX_INTRAPRED

          //在不是MRL模式时将两轮SATD后更新好的列表保存下来，存到m_rdModeListWithoutMrl中（没有MPM模式）
          m_rdModeListWithoutMrl = uiRdModeList;
#endif
        }
#if ENABLE_JVET_L0283_MRL
        pu.multiRefIdx = 1;//使用1参考行
        const int  numMPMs = NUM_MOST_PROBABLE_MODES;//定义最可能模式的数量，即MPM列表的大小
        unsigned  multiRefMPM [numMPMs];//定义多参考行的MPM列表
        PU::getIntraMPMs(pu, multiRefMPM);//获取该MPM列表的函数入口

        //对每个扩展的参考行进行循环
        for (int mRefNum = 1; mRefNum < numOfPassesExtendRef; mRefNum++)
        {

          //定义多参考行的索引，不同参考行的MPM列表是不同的
          int multiRefIdx = MULTI_REF_LINE_IDX[mRefNum];

          pu.multiRefIdx = multiRefIdx;
          {
            initIntraPatternChType(cu, pu.Y(), true);
          }
#if JVET_N0185_UNIFIED_MPM
          //对MPM列表中的6个模式进行SATD比较选择
          for (int x = 1; x < numMPMs; x++)
#endif
          {
            uint32_t mode = multiRefMPM[x];
            {
              pu.intraDir[0] = mode;
              initPredIntraParams(pu, pu.Y(), sps);

              if (useDPCMForFirstPassIntraEstimation(pu, mode))
              {
                encPredIntraDPCM(COMPONENT_Y, piOrg, piPred, mode);
              }
              else
              {
                predIntraAng(COMPONENT_Y, piPred, pu);
              }

#if JVET_N0363_INTRA_COST_MOD
              // Use the min between SAD and SATD as the cost criterion
              // SAD is scaled by 2 to align with the scaling of HAD
              Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));
#endif

              // NB xFracModeBitsIntra will not affect the mode for chroma that may have already been pre-estimated.
#if JVET_N0217_MATRIX_INTRAPRED
              m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
#endif
              m_CABACEstimator->getCtx() = SubCtx( Ctx::ISPMode, ctxStartIspMode );
#if JVET_N0185_UNIFIED_MPM
              m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
              m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
#if !JVET_N0302_SIMPLFIED_CIIP
              m_CABACEstimator->getCtx() = SubCtx( Ctx::MHIntraPredMode, ctxStartMHIntraMode );
#endif
              m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );

              uint64_t fracModeBits = xFracModeBitsIntra(pu, mode, CHANNEL_TYPE_LUMA);

#if JVET_N0363_INTRA_COST_MOD
              double cost = (double)minSadHad + (double)fracModeBits * sqrtLambdaForFirstPass;

#endif
#if JVET_N0217_MATRIX_INTRAPRED
              updateCandList( ModeInfo( false, multiRefIdx, NOT_INTRA_SUBPARTITIONS, mode ), cost,        uiRdModeList,  CandCostList, numModesForFullRD );
#if JVET_N0363_INTRA_COST_MOD

              //更新候选列表
              updateCandList( ModeInfo( false, multiRefIdx, NOT_INTRA_SUBPARTITIONS, mode ), (double)minSadHad, uiHadModeList, CandHadList,  numHadCand );     
#endif
#else
        
#endif
            }
          }
        }
#endif
#if JVET_N0217_MATRIX_INTRAPRED
        CHECKD( uiRdModeList.size() != numModesForFullRD, "Error: RD mode list size" );

          //*** Derive MIP candidates using Hadamard
        //在两轮传统角度模式的SATD以及MPM的SATD结束以后，对使用哈达吗的MIp模式单独进行候选模式的推导
          if (testMip)
          {
            cu.mipFlag = true;
            pu.multiRefIdx = 0;
            //初始化Mip模式
            initIntraMip( pu );

            //getNumModesMip函数根据当前CU的块尺寸获取不同的Mip模式数量
            for (uint32_t uiMode = 0; uiMode < getNumModesMip(pu.Y()); uiMode++)
            {
              pu.intraDir[CHANNEL_TYPE_LUMA] = uiMode;

              //Mip预测模式的函数入口
              predIntraMip(COMPONENT_Y, piPred, pu);

#if JVET_N0363_INTRA_COST_MOD
              // Use the min between SAD and HAD as the cost criterion
              // SAD is scaled by 2 to align with the scaling of HAD
              Distortion minSadHad = std::min(distParamSad.distFunc(distParamSad)*2, distParamHad.distFunc(distParamHad));

#endif

              m_CABACEstimator->getCtx() = SubCtx( Ctx::MipFlag, ctxStartMipFlag );
              m_CABACEstimator->getCtx() = SubCtx( Ctx::MipMode, ctxStartMipMode );

              uint64_t fracModeBits = xFracModeBitsIntra(pu, uiMode, CHANNEL_TYPE_LUMA);

#if JVET_N0363_INTRA_COST_MOD
              double cost = double(minSadHad) + double(fracModeBits) * sqrtLambdaForFirstPass;

#endif
              //更新候选列表，将最优的Mip模式加入到RDCost的候选列表中
              updateCandList(ModeInfo(true, 0, NOT_INTRA_SUBPARTITIONS, uiMode),        cost,  uiRdModeList,  CandCostList, numModesForFullRD);
#if JVET_N0363_INTRA_COST_MOD
              updateCandList(ModeInfo(true, 0, NOT_INTRA_SUBPARTITIONS, uiMode), double(minSadHad), uiHadModeList, CandHadList, numHadCand);

#endif
            }


            //在总共四轮的STAD之后，最终更新好RDcost列表后，还要对该列表进行一个缩减的处理
            const double thresholdHadCost = 1.0 + 1.4 / sqrt((double)(pu.lwidth()*pu.lheight()));//计算代价的阈值
            reduceHadCandList(uiRdModeList, CandCostList, numModesForFullRD, thresholdHadCost, 0.0);//缩减列表的函数入口


          }

#endif
        //获取FastUDI可能的MPM列表，如果满足条件也将其加入RD候选列表
        if( m_pcEncCfg->getFastUDIUseMPMEnabled() )
        {
          const int numMPMs = NUM_MOST_PROBABLE_MODES;
          unsigned  uiPreds[numMPMs];

          pu.multiRefIdx = 0;

          const int numCand = PU::getIntraMPMs( pu, uiPreds );//定义MPM列表中最可能的模式数量,然后只对MPM中最可能的那几个模式进行SATD粗选，其余模式不进行

          for( int j = 0; j < numCand; j++ )
          {
            bool mostProbableModeIncluded = false;
#if JVET_N0217_MATRIX_INTRAPRED
            ModeInfo mostProbableMode( false, 0, NOT_INTRA_SUBPARTITIONS, uiPreds[j] );
#else
            int  mostProbableMode         = uiPreds[j];
#endif


            for( int i = 0; i < numModesForFullRD; i++ )
            {
#if JVET_N0217_MATRIX_INTRAPRED
              mostProbableModeIncluded |= ( mostProbableMode == uiRdModeList[i] );//如果不包括在最可能模式中，则将其放入RD候选列表中
#else
              mostProbableModeIncluded |= (mostProbableMode == uiRdModeList[i] && extendRefList[i] == 0);
#endif
            }
            if( !mostProbableModeIncluded )
            {
#if !JVET_N0217_MATRIX_INTRAPRED
              extendRefList.push_back(0);
#endif
              numModesForFullRD++;
              uiRdModeList.push_back( mostProbableMode );
#if JVET_N0217_MATRIX_INTRAPRED
              CandCostList.push_back(0);
#endif
            }
          }
          if( nOptionsForISP > 1 )
          {
            //we add the ISP MPMs to the list without mrl modes
            //将不加入多参考情况的ISP模式添加到RD候选列表中
            m_rdModeListWithoutMrlHor = m_rdModeListWithoutMrl;
            m_rdModeListWithoutMrlVer = m_rdModeListWithoutMrl;
#if JVET_N0217_MATRIX_INTRAPRED
            for (int k = 0; k < m_rdModeListWithoutMrl.size(); k++)
            {
              m_rdModeListWithoutMrlHor[k].ispMod = HOR_INTRA_SUBPARTITIONS;
              m_rdModeListWithoutMrlVer[k].ispMod = VER_INTRA_SUBPARTITIONS;
            }

            //定义一个指示器列表，存入非MRL的ISP模式
            static_vector* listPointer;

#endif
			//对每一个ISP的分区进行STAD的粗选
            for( int k = 1; k < nOptionsForISP; k++ )
            {
              cu.ispMode = ispOptions[k];
              listPointer = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? m_rdModeListWithoutMrlHor : m_rdModeListWithoutMrlVer );
              const int numCandISP = PU::getIntraMPMs( pu, uiPreds );//这里再定义一个针对ISP模式的最可能的MPM模式数量
              for( int j = 0; j < numCandISP; j++ )
              {
                bool mostProbableModeIncluded = false;
#if JVET_N0217_MATRIX_INTRAPRED
                ModeInfo mostProbableMode( false, 0, cu.ispMode, uiPreds[j] );

#endif

                for( int i = 0; i < listPointer->size(); i++ )
                {
                  mostProbableModeIncluded |= ( mostProbableMode == listPointer->at( i ) );
                }
                if( !mostProbableModeIncluded )
                {
                  listPointer->push_back( mostProbableMode );//如果该模式不包括在最可能模式中，将ISP的最可能模式push到listPointer中
                }
              }
            }
            cu.ispMode = NOT_INTRA_SUBPARTITIONS;
          }
        }
#if JVET_N0217_MATRIX_INTRAPRED
        //*** Add MPMs for MIP to candidate list
        //将Mip模式的MPM列表添加到RD候选列表中
        if (!fastMip && testMip && pu.lwidth() < 8 && pu.lheight() < 8)//快速算法的Mip不参与其中
        {
          //定义一个Mip模式专属的MPM列表，大小为3
          unsigned mpm[NUM_MPM_MIP];

          //获取当前PU的Mip模式的MPM列表
          int numCandMip = PU::getMipMPMs(pu, mpm);

          for( int j = 0; j < numCandMip; j++ )
          {
            bool mostProbableModeIncluded = false;

            //判断MPM列表中的模式是否包括在最可能模式之中，如果不包括，则将该模式加入到RD候选列表中
            ModeInfo mostProbableMode(true, 0, NOT_INTRA_SUBPARTITIONS, mpm[j]);
            for( int i = 0; i < numModesForFullRD; i++ )
            {
              mostProbableModeIncluded |= (mostProbableMode == uiRdModeList[i]);
            }
            if( !mostProbableModeIncluded )
            {
              numModesForFullRD++;
              uiRdModeList.push_back( mostProbableMode );
              CandCostList.push_back(0);
            }
          }
        }
#endif
      }
      else
      {
#if JVET_N0217_MATRIX_INTRAPRED
        THROW( "Full search not supported for MIP" );
#endif
      }
#if JVET_N0193_LFNST
      if( sps.getUseLFNST() && mtsUsageFlag == 1 )
      {
        // Store the modes to be checked with RD

        //保存最终用于RDCost的候选模式数量
        m_savedNumRdModes[ lfnstIdx ]     = numModesForFullRD;
        std::copy_n( uiRdModeList.begin(),  numModesForFullRD, m_savedRdModeList[ lfnstIdx ] );

      }
#endif
    }
#if JVET_N0193_LFNST
    else //mtsUsage = 2 (here we potentially reduce the number of modes that will be full-RD checked)
    {

      //如果使用帧内快速LFNST
      if( m_pcEncCfg->getUseFastLFNST() || !cu.slice->isIntra() )
      {

        //重置RD候选模式列表的数量，下面进行第二次RD候选列表的缩减
        numModesForFullRD = 0;


        //设置一个跳过模式阈值
        double thresholdSkipMode = 1.0 + ( ( cu.lfnstIdx > 0 ) ? 0.1 : 1.0 ) * ( 1.4 / sqrt( ( double ) ( width*height ) ) );

        //Skip checking the modes with much larger R-D cost than the best mode
        //对所有RD候选模式进行循环处理，但是跳过检查比最佳模式代价大得多的R-D模式，不进行RDcost检查
        for( int i = 0; i < m_savedNumRdModes[ lfnstIdx ]; i++ )
        {
          //如果该模式的代价小于阈值，则将其保留在RD候选模式列表中，否则将其排除
          if( m_modeCostStore[ lfnstIdx ][ i ] <= thresholdSkipMode * m_bestModeCostStore[ lfnstIdx ] )
          {
            uiRdModeList.push_back( m_savedRdModeList[ lfnstIdx ][ i ] );

            //RD候选模式列表的数量加一
            numModesForFullRD++;
          }
        }
      }
      else //this is necessary because we skip the candidates list calculation, since it was already obtained for the DCT-II. Now we load it
      //这里的操作是必要的，因为已经从DCT-2中获得了计算数据，因此这里跳过了候选列表的计算，现在我们加载它
      {
        // Restore the modes to be checked with RD 
        // 重新保存用于RD的候选模式数量
        numModesForFullRD = m_savedNumRdModes[ lfnstIdx ];
        //重置RD候选列表的大小
        uiRdModeList.resize( numModesForFullRD );
        std::copy_n( m_savedRdModeList[ lfnstIdx ], m_savedNumRdModes[ lfnstIdx ], uiRdModeList.begin() );

        CandCostList.resize( numModesForFullRD );

      }
    }
#endif

    if( nOptionsForISP > 1 ) // we remove the non-MPMs from the ISP lists
      //从ISP列表中移除非MPM模式
    {
#if JVET_N0217_MATRIX_INTRAPRED
      static_vector uiRdModeListCopyHor = m_rdModeListWithoutMrlHor;
      m_rdModeListWithoutMrlHor.clear();
      static_vector uiRdModeListCopyVer = m_rdModeListWithoutMrlVer;
      m_rdModeListWithoutMrlVer.clear();
      static_vector *listPointerCopy, *listPointer;

#endif
      for( int ispOptionIdx = 1; ispOptionIdx < nOptionsForISP; ispOptionIdx++ )
      {
        cu.ispMode = ispOptions[ispOptionIdx];
        //we get the mpm cand list
        //获取对于ISP模式的MPM的候选列表
        const int numMPMs = NUM_MOST_PROBABLE_MODES;
        unsigned  uiPreds[numMPMs];

        pu.multiRefIdx = 0;

        PU::getIntraMPMs( pu, uiPreds );

        //we copy only the ISP MPMs
        //仅仅复制ISP的MPM候选模式
        listPointerCopy = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? uiRdModeListCopyHor : uiRdModeListCopyVer );
        listPointer     = &( cu.ispMode == HOR_INTRA_SUBPARTITIONS ? m_rdModeListWithoutMrlHor : m_rdModeListWithoutMrlVer );
        
        //对ISP列表的重置的循环操作
        for( int k = 0; k < listPointerCopy->size(); k++ )
        {
          for( int q = 0; q < numMPMs; q++ )
          {
#if JVET_N0217_MATRIX_INTRAPRED
            if (listPointerCopy->at(k) == ModeInfo( false, 0, cu.ispMode, uiPreds[q] ))

#endif
            {
              listPointer->push_back( listPointerCopy->at( k ) );
              break;
            }
          }
        }
      }
      cu.ispMode = NOT_INTRA_SUBPARTITIONS;
    }


    CHECK( numModesForFullRD != uiRdModeList.size(), "Inconsistent state!" );

    // after this point, don't use numModesForFullRD
    //这一步操作之后，先不要去使用numModesForFullRD作为最终的RD候选列表的大小


    // PBINTRA fast快速帧内模式
#if JVET_N0193_LFNST
#if JVET_N0329_IBC_SEARCH_IMP
    if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && !cs.slice->getDisableSATDForRD() && ( mtsUsageFlag != 2 || lfnstIdx > 0 ) )
#else
    if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && ( mtsUsageFlag != 2 || lfnstIdx > 0 ) )
#endif
#else
#if JVET_N0329_IBC_SEARCH_IMP
    if (m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable && !cs.slice->getDisableSATDForRD())
#else
    if( m_pcEncCfg->getUsePbIntraFast() && !cs.slice->isIntra() && uiRdModeList.size() < numModesAvailable )
#endif
#endif
    {
#if JVET_N0217_MATRIX_INTRAPRED
#if JVET_N0193_LFNST
      double pbintraRatio = (lfnstIdx > 0) ? 1.25 : PBINTRA_RATIO;
#endif
      int maxSize = -1;
      const int numHadCand = (testMip ? 2 : 1) * 3;
      for (int k = numHadCand - 1; k >= 0; k--)
      {
#if JVET_N0193_LFNST
        if (CandHadList.size() < (k + 1) || CandHadList[k] > cs.interHad * pbintraRatio) { maxSize = k; }
#else
        if (CandHadList.size() < (k + 1) || CandHadList[k] > cs.interHad * PBINTRA_RATIO) { maxSize = k; }
#endif
      }
      if (maxSize > 0)
      {
        uiRdModeList.resize(std::min(uiRdModeList.size(), maxSize));
        if (nOptionsForISP > 1)
        {
          m_rdModeListWithoutMrlHor.resize(std::min(m_rdModeListWithoutMrlHor.size(), maxSize));
          m_rdModeListWithoutMrlVer.resize(std::min(m_rdModeListWithoutMrlVer.size(), maxSize));
        }
      }
      if (maxSize == 0)
      {
        cs.dist = std::numeric_limits::max();
        cs.interHad = 0;

        //===== reset context models =====
        m_CABACEstimator->getCtx() = SubCtx(Ctx::MipFlag, ctxStartMipFlag);
        m_CABACEstimator->getCtx() = SubCtx(Ctx::MipMode, ctxStartMipMode);
        m_CABACEstimator->getCtx() = SubCtx(Ctx::ISPMode, ctxStartIspMode);
#if JVET_N0185_UNIFIED_MPM
        m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
        m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaMpmFlag, ctxStartIntraMode);
#if !JVET_N0302_SIMPLFIED_CIIP
        m_CABACEstimator->getCtx() = SubCtx(Ctx::MHIntraPredMode, ctxStartMHIntraMode);
#endif
        m_CABACEstimator->getCtx() = SubCtx(Ctx::MultiRefLineIdx, ctxStartMrlIdx);

#if JVET_N0193_LFNST
        return false;
#else
        return;
#endif
      }
#else
#if JVET_N0193_LFNST
      double pbintraRatio = ( lfnstIdx > 0 ) ? 1.25 : PBINTRA_RATIO;
      if( CandHadList.size() < 3 || CandHadList[ 2 ] > cs.interHad * pbintraRatio )
#else
      if( CandHadList.size() < 3 || CandHadList[2] > cs.interHad * PBINTRA_RATIO )
#endif
      {
        uiRdModeList.resize( std::min( uiRdModeList.size(), 2 ) );
#if !JVET_N0217_MATRIX_INTRAPRED
        extendRefList.resize( std::min( extendRefList.size(), 2 ) );
#endif
        if( nOptionsForISP > 1 )
        {
          m_rdModeListWithoutMrlHor.resize( std::min( m_rdModeListWithoutMrlHor.size(), 2 ) );
          m_rdModeListWithoutMrlVer.resize( std::min( m_rdModeListWithoutMrlVer.size(), 2 ) );
        }
      }
#if JVET_N0193_LFNST
      if( CandHadList.size() < 2 || CandHadList[ 1 ] > cs.interHad * pbintraRatio )
#else
      if( CandHadList.size() < 2 || CandHadList[1] > cs.interHad * PBINTRA_RATIO )
#endif
      {
        uiRdModeList.resize( std::min( uiRdModeList.size(), 1 ) );
#if !JVET_N0217_MATRIX_INTRAPRED
        extendRefList.resize( std::min( extendRefList.size(), 1 ) );
#endif
        if( nOptionsForISP > 1 )
        {
          m_rdModeListWithoutMrlHor.resize( std::min( m_rdModeListWithoutMrlHor.size(), 1 ) );
          m_rdModeListWithoutMrlVer.resize( std::min( m_rdModeListWithoutMrlVer.size(), 1 ) );
        }
      }
#if JVET_N0193_LFNST
      if( CandHadList.size() < 1 || CandHadList[ 0 ] > cs.interHad * pbintraRatio )
#else
      if( CandHadList.size() < 1 || CandHadList[0] > cs.interHad * PBINTRA_RATIO )
#endif
      {
        cs.dist = std::numeric_limits::max();
        cs.interHad = 0;

        //===== reset context models =====
        m_CABACEstimator->getCtx() = SubCtx(Ctx::ISPMode, ctxStartIspMode);
#if JVET_N0185_UNIFIED_MPM
        m_CABACEstimator->getCtx() = SubCtx(Ctx::IntraLumaPlanarFlag, ctxStartPlanarFlag);
#endif
        m_CABACEstimator->getCtx() = SubCtx( Ctx::IntraLumaMpmFlag, ctxStartIntraMode );
#if !JVET_N0302_SIMPLFIED_CIIP
        m_CABACEstimator->getCtx() = SubCtx( Ctx::MHIntraPredMode, ctxStartMHIntraMode );
#endif
        m_CABACEstimator->getCtx() = SubCtx( Ctx::MultiRefLineIdx, ctxStartMrlIdx );

#if JVET_N0193_LFNST
        return false;
#else
        return;
#endif
      }
#endif
    }

    if ( nOptionsForISP > 1 )
    {
      //we create a single full RD list that includes all intra modes using regular intra, MRL and ISP
      //创造一个完整的RD候选列表，其中包括使用常规intra、mrl和isp的所有内部模式。
      auto* firstIspList  = ispOptions[1] == HOR_INTRA_SUBPARTITIONS ? &m_rdModeListWithoutMrlHor : &m_rdModeListWithoutMrlVer;
      auto* secondIspList = ispOptions[1] == HOR_INTRA_SUBPARTITIONS ? &m_rdModeListWithoutMrlVer : &m_rdModeListWithoutMrlHor;

#if JVET_N0193_LFNST
      if( !sps.getUseLFNST() && m_pcEncCfg->getUseFastISP() )//使用快速ISP模式
#else
      if ( m_pcEncCfg->getUseFastISP() )
#endif
      {
#if JVET_N0217_MATRIX_INTRAPRED
        CHECKD( uiRdModeList.size() > CandCostList.size(), "Error: CandCostList size" );
        // find the first non-MRL, non-MIP mode
        //找到第一个非MRL和非Mip模式
        int indexFirstMode = int(uiRdModeList.size()) - 1; // default is last mode默认是最后一个模式

        //对RD候选模式列表中的每个模式进行循环遍历处理
        for (int k = 0; k < int(uiRdModeList.size()); k++)
        {
          if (uiRdModeList[k].mRefId == 0 && uiRdModeList[k].mipFlg == false)
          {
            indexFirstMode = k;
            break;
          }
        }
        // move the mode indicated by indexFirstMode to the beginning
        //将indexFirstMode指示的模式移到候选列表的开头
        for (int idx = indexFirstMode - 1; idx >= 0; idx--)
        {
          std::swap(uiRdModeList[idx], uiRdModeList[idx + 1]);
          std::swap(CandCostList[idx], CandCostList[idx + 1]);
        }
        //insert all ISP modes after the first non-mrl mode
        //然后在第一个非MRL模式之后插入所有的ISP模式
        uiRdModeList.insert(uiRdModeList.begin() + 1, secondIspList->begin(), secondIspList->end());
        uiRdModeList.insert(uiRdModeList.begin() + 1, firstIspList->begin(), firstIspList->end());
#else
        // find the first non-MRL mode
        size_t indexFirstMode = std::find( extendRefList.begin(), extendRefList.end(), 0 ) - extendRefList.begin();
        // if not found, just take the last mode
        if( indexFirstMode >= extendRefList.size() ) indexFirstMode = extendRefList.size() - 1;
        // move the mode indicated by indexFirstMode to the beginning
        for( int idx = ((int)indexFirstMode) - 1; idx >= 0; idx-- )
        {
          std::swap( extendRefList[idx], extendRefList[idx + 1] );
          std::swap( uiRdModeList [idx], uiRdModeList [idx + 1] );
        }
        //insert all ISP modes after the first non-mrl mode
        uiRdModeList.insert( uiRdModeList.begin() + 1, secondIspList->begin(), secondIspList->end() );
        uiRdModeList.insert( uiRdModeList.begin() + 1, firstIspList->begin() , firstIspList->end()  );

        extendRefList.insert( extendRefList.begin() + 1, secondIspList->size(), MRL_NUM_REF_LINES + ispOptions[2] );
        extendRefList.insert( extendRefList.begin() + 1, firstIspList->size() , MRL_NUM_REF_LINES + ispOptions[1] );
#endif
      }
      else
      {
        //insert all ISP modes at the end of the current list
        //在当前的列表的结尾插入所有的ISP模式
        uiRdModeList.insert( uiRdModeList.end(), secondIspList->begin(), secondIspList->end() );
        uiRdModeList.insert( uiRdModeList.end(), firstIspList->begin() , firstIspList->end()  );
#if !JVET_N0217_MATRIX_INTRAPRED
        extendRefList.insert( extendRefList.end(), secondIspList->size(), MRL_NUM_REF_LINES + ispOptions[2] );
        extendRefList.insert( extendRefList.end(), firstIspList->size() , MRL_NUM_REF_LINES + ispOptions[1] );
#endif
      }
    }
#if !JVET_N0217_MATRIX_INTRAPRED
    CHECKD(uiRdModeList.size() != extendRefList.size(),"uiRdModeList and extendRefList do not have the same size!");
#endif

    //===== check modes (using r-d costs) =====
    //check 所有模式（使用RDcost）
#if JVET_N0217_MATRIX_INTRAPRED
    ModeInfo       uiBestPUMode;//定义当前PU最优的亮度预测帧内模式
#else
    uint32_t       uiBestPUMode  = 0;
    int            bestExtendRef = 0;
#endif
#if JVET_N0413_RDPCM
    int            bestBDPCMMode = 0;
    double         bestCostNonBDPCM = MAX_DOUBLE;//定义再不使用BDPCM模式的时候的最优代价
#endif


    //临时缓存CS
    CodingStructure *csTemp = m_pTempCS[gp_sizeIdxInfo->idxFrom( cu.lwidth() )][gp_sizeIdxInfo->idxFrom( cu.lheight() )];
    
    //最好的模式缓存CS
    CodingStructure *csBest = m_pBestCS[gp_sizeIdxInfo->idxFrom( cu.lwidth() )][gp_sizeIdxInfo->idxFrom( cu.lheight() )];

    csTemp->slice = cs.slice;
    csBest->slice = cs.slice;
    csTemp->initStructData();
    csBest->initStructData();

#if JVET_N0217_MATRIX_INTRAPRED

    //定义非Mip模式的最优的代价
    m_bestCostNonMip = MAX_DOUBLE;
#if JVET_N0193_LFNST

    //重新定义一个RD候选列表
    static_vector rdModeIdxList;
#endif
    if (testMip)
    {

      //如果是Mip模式的时候，单独为其设置ui的RD候选临时列表
    static_vector uiRdModeListTemp;

    //对RD候选列表中的模式进行循环遍历处理
    //先把非ISP且非MIP的模式放到重新定义的RD候选列表的前面
    for( int i = 0; i < uiRdModeList.size(); i++)
    {
      //如果不是ISP模式且不是MIp模式
      if( !uiRdModeList[i].mipFlg && uiRdModeList[i].ispMod==NOT_INTRA_SUBPARTITIONS )
      {
        uiRdModeListTemp.push_back( uiRdModeList[i] );//将SAD之后的RD候选模式列表中的模式push到临时列表中保存起来
#if JVET_N0193_LFNST
        rdModeIdxList.push_back( i );//给重新定义的RD候选列表中push对应的在原来RD候选模式列表中的模式号
#endif
      }
    }

    //再把ISP或者MIP的模式放到重新定义的RD候选列表的后面
    for( int i = 0; i < uiRdModeList.size(); i++)
    {
      //如果是Mip模式或者是ISP模式
      if( uiRdModeList[i].mipFlg || uiRdModeList[i].ispMod!=NOT_INTRA_SUBPARTITIONS )
      {
        uiRdModeListTemp.push_back( uiRdModeList[i] );//也将SAD之后的RD候选模式列表中的模式push到临时列表中保存起来
#if JVET_N0193_LFNST
        rdModeIdxList.push_back( i );//给重新定义的RD候选列表中push对应的在原来RD候选模式列表中的模式号
#endif
      }
    }


    //最后将原来的RD候选模式列表恢复出来
    for( int i = 0; i < uiRdModeList.size(); i++)
    {
      uiRdModeList[i] = uiRdModeListTemp[i];//
    }
    }
#endif
    // just to be sure
    //确保numModesForFullRD和RD候选列表数目相等
    numModesForFullRD = ( int ) uiRdModeList.size();
    PartSplit intraSubPartitionsProcOrder = TU_NO_ISP;
    int       bestNormalIntraModeIndex    = -1;
#if !JVET_N0217_MATRIX_INTRAPRED
    uint8_t   bestIspOption               = NOT_INTRA_SUBPARTITIONS;
#endif
    TUIntraSubPartitioner subTuPartitioner( partitioner );
#if JVET_N0193_LFNST
    if( !cu.ispMode && !cu.mtsFlag )
    {
      m_modeCtrl->setMtsFirstPassNoIspCost( MAX_DOUBLE );
    }
#endif
    bool      ispHorAllZeroCbfs = false, ispVerAllZeroCbfs = false;

#if JVET_N0217_MATRIX_INTRAPRED
#if JVET_N0413_RDPCM

    /********************这里才进入到最终对于RD候选列表中的模式的进一步精选RDcost操作***********/
    for (int mode = -2 * int(testBDPCM); mode < (int)uiRdModeList.size(); mode++)
    {
      // set CU/PU to luma prediction mode
      //首先设置亮度CU/PU的预测模式
      ModeInfo uiOrgMode;

      //如果是BDPCM模式
      if ( mode < 0 )
      {
        //将负的模式号转化为正的
        cu.bdpcmMode = -mode;
        //定义一个MPM列表
        unsigned mpm_pred[NUM_MOST_PROBABLE_MODES];
        PU::getIntraMPMs(pu, mpm_pred);

        uiOrgMode = ModeInfo(false, 0, NOT_INTRA_SUBPARTITIONS, mpm_pred[0]);
        cu.mipFlag                     = uiOrgMode.mipFlg;
        cu.ispMode                     = uiOrgMode.ispMod;
        pu.multiRefIdx                 = uiOrgMode.mRefId;
        pu.intraDir[CHANNEL_TYPE_LUMA] = uiOrgMode.modeId;
      }
      else//如果不是BDPCM模式
      {
        cu.bdpcmMode = 0;
        uiOrgMode = uiRdModeList[mode];//取出RD候选列表中的模式
#else
    for (uint32_t uiMode = 0; uiMode < uiRdModeList.size(); uiMode++)
    {
      // set CU/PU to luma prediction mode
      ModeInfo uiOrgMode = uiRdModeList[uiMode];
#endif
      cu.mipFlag                     = uiOrgMode.mipFlg;//当前候选模式是否是Mip模式
      cu.ispMode                     = uiOrgMode.ispMod;//当前候选模式是否是ISP模式
      pu.multiRefIdx                 = uiOrgMode.mRefId;//当前候选模式是否是使用MRL
      pu.intraDir[CHANNEL_TYPE_LUMA] = uiOrgMode.modeId;//当前候选模式放入intraDir

      CHECK(cu.mipFlag && pu.multiRefIdx, "Error: combination of MIP and MRL not supported");//MIP and MRL的结合不被允许
#if JVET_N0185_UNIFIED_MPM
      CHECK(pu.multiRefIdx && (pu.intraDir[0] == PLANAR_IDX), "Error: combination of MRL and Planar mode not supported");//Planar and MRL的结合不被允许
#else
      CHECK(pu.multiRefIdx && (pu.intraDir[0] == DC_IDX || pu.intraDir[0] == PLANAR_IDX), "Error: combination of MRL and Planar/DC mode not supported");
#endif
      CHECK(cu.ispMode && cu.mipFlag, "Error: combination of ISP and MIP not supported");//ISP and MIP的结合不被允许
      CHECK(cu.ispMode && pu.multiRefIdx, "Error: combination of ISP and MRL not supported");//ISP and MRL的结合不被允许
#else
#if JVET_N0413_RDPCM
    for( int mode = -2 * int(testBDPCM); mode < numModesForFullRD; mode++ )
#else
    for (uint32_t uiMode = 0; uiMode < numModesForFullRD; uiMode++)
#endif
    {
#if JVET_N0413_RDPCM
      int multiRefIdx = 0;
      uint32_t uiOrgMode;

      if ( mode < 0 )
    {
        cu.bdpcmMode = -mode;
        unsigned mpm_pred[NUM_MOST_PROBABLE_MODES];
        PU::getIntraMPMs(pu, mpm_pred);
        pu.intraDir[0] = mpm_pred[0];
        uiOrgMode = mpm_pred[0];
        cu.ispMode = NOT_INTRA_SUBPARTITIONS;
      }
      else
      {
        cu.bdpcmMode = 0;
        uiOrgMode = uiRdModeList[mode];
#else
      // set luma prediction mode
      uint32_t uiOrgMode = uiRdModeList[uiMode];
#endif
      cu.ispMode = extendRefList[mode] > MRL_NUM_REF_LINES ? extendRefList[mode] - MRL_NUM_REF_LINES : NOT_INTRA_SUBPARTITIONS;
        pu.intraDir[0] = uiOrgMode;

#if !JVET_N0413_RDPCM
        int multiRefIdx = 0;
#endif
        pu.multiRefIdx = multiRefIdx;
#endif
        if( cu.ispMode )//如果当前CU是ISP模式
        {
          intraSubPartitionsProcOrder = CU::getISPType( cu, COMPONENT_Y );
          bool tuIsDividedInRows = CU::divideTuInRows( cu );//TU是否进行行划分
          if ( ( tuIsDividedInRows && ispHorAllZeroCbfs ) || ( !tuIsDividedInRows && ispVerAllZeroCbfs ) )
          {
            continue;
          }
          if( m_intraModeDiagRatio.at( bestNormalIntraModeIndex ) > 1.25 )
          {
            continue;
          }
          if( ( m_intraModeHorVerRatio.at( bestNormalIntraModeIndex ) > 1.25 && tuIsDividedInRows ) || ( m_intraModeHorVerRatio.at( bestNormalIntraModeIndex ) < 0.8 && !tuIsDividedInRows ) )
          {
            continue;
          }
        }

#if JVET_N0413_RDPCM
      }
#endif

      // set context models
	//设置上下文模型
      m_CABACEstimator->getCtx() = ctxStart;

      // determine residual for partition
	  //确定子分区的残差
      cs.initSubStructure( *csTemp, partitioner.chType, cs.area, true );

#if JVET_N0193_LFNST
      bool tmpValidReturn = false;
#endif
      if( cu.ispMode )//如果当前CU是ISP模式，则对每一个子分区用当前模式进行亮度预测
      {
#if JVET_N0193_LFNST
		// 该函数主要用于计算预测值以及残差值，然后变换量化并且计算重建值，最后计算率失真代价
        tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, subTuPartitioner, bestCurrentCost, 0, intraSubPartitionsProcOrder, false,
                                                  mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
        xRecurIntraCodingLumaQT( *csTemp, subTuPartitioner, bestCurrentCost, 0, intraSubPartitionsProcOrder );
#endif
      }
      else//如果不是ISP模式，则正常进行帧内预测
      {
#if JVET_N0217_MATRIX_INTRAPRED
        if( ! fastMip )
        {
          m_bestCostNonMip = MAX_DOUBLE;
        }
#if JVET_N0193_LFNST
		//帧内亮度预测主函数的入口
        tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, partitioner, uiBestPUMode.ispMod ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, uiBestPUMode.ispMod,
                                                  mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
        xRecurIntraCodingLumaQT(*csTemp, partitioner, uiBestPUMode.ispMod ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, uiBestPUMode.ispMod);
#endif
#else
#if JVET_N0193_LFNST
        tmpValidReturn = xRecurIntraCodingLumaQT( *csTemp, partitioner, bestIspOption ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, bestIspOption,
                                                  mtsCheckRangeFlag, mtsFirstCheckId, mtsLastCheckId, moreProbMTSIdxFirst );
#else
        xRecurIntraCodingLumaQT( *csTemp, partitioner, bestIspOption ? bestCurrentCost : MAX_DOUBLE, -1, TU_NO_ISP, bestIspOption );
#endif
#endif
      }
	  //如果当前CU是ISP模式，且第一块TU不是亮度TU
      if( cu.ispMode && !csTemp->cus[0]->firstTU->cbf[COMPONENT_Y] )
      {
#if JVET_N0193_LFNST
        if( !sps.getUseLFNST() )
        {
#endif
          if ( cu.ispMode == HOR_INTRA_SUBPARTITIONS )
          {
            ispHorAllZeroCbfs |= ( m_pcEncCfg->getUseFastISP() && csTemp->tus[0]->lheight() > 2 && csTemp->cost >= bestCurrentCost );
          }
          else
          {
            ispVerAllZeroCbfs |= ( m_pcEncCfg->getUseFastISP() && csTemp->tus[0]->lwidth() > 2 && csTemp->cost >= bestCurrentCost );
          }
#if JVET_N0193_LFNST
        }
#endif
        csTemp->cost = MAX_DOUBLE;
        csTemp->costDbOffset = 0;
#if JVET_N0193_LFNST
        tmpValidReturn = false;
#endif
      }
#if JVET_N0193_LFNST
      validReturn |= tmpValidReturn;

#if JVET_N0413_RDPCM
      if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode && mode >= 0 )
      {
#if JVET_N0217_MATRIX_INTRAPRED
        m_modeCostStore[ lfnstIdx ][ testMip ? rdModeIdxList[ mode ] : mode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#else
        m_modeCostStore[ lfnstIdx ][ mode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#endif
#else
      if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode )
      {
#if JVET_N0217_MATRIX_INTRAPRED
        m_modeCostStore[ lfnstIdx ][ testMip ? rdModeIdxList[ uiMode ] : uiMode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#else
        m_modeCostStore[ lfnstIdx ][ uiMode ] = tmpValidReturn ? csTemp->cost : ( MAX_DOUBLE / 2.0 ); //(MAX_DOUBLE / 2.0) ??
#endif
#endif
      }
#endif


#if JVET_N0217_MATRIX_INTRAPRED
      DTRACE( g_trace_ctx, D_INTRA_COST, "IntraCost T %f (%d) \n", csTemp->cost, uiOrgMode.modeId );
#else
      DTRACE( g_trace_ctx, D_INTRA_COST, "IntraCost T %f (%d) \n", csTemp->cost, uiOrgMode );
#endif


#if JVET_N0193_LFNST
      if( tmpValidReturn )
      {
#endif
        // check r-d cost

		  //check 每个模式的RD cost，逐次筛选出cost最小的模式
        if( csTemp->cost < csBest->cost )//如果当前模式的cost小于目前最优的模式的cost
        {
          std::swap( csTemp, csBest );//当前模式成为最新的最优模式

          uiBestPUMode  = uiOrgMode;//将当前模式赋值给最优模式uiBestPUMode
#if !JVET_N0217_MATRIX_INTRAPRED
          bestExtendRef = multiRefIdx;
          bestIspOption = cu.ispMode;
#endif
#if JVET_N0413_RDPCM
          bestBDPCMMode = cu.bdpcmMode;
#endif
#if JVET_N0193_LFNST
          if( sps.getUseLFNST() && mtsUsageFlag == 1 && !cu.ispMode )
          {
            m_bestModeCostStore[ lfnstIdx ] = csBest->cost; //cs.cost;
          }
#endif
          if( csBest->cost < bestCurrentCost )
          {
            bestCurrentCost = csBest->cost;
          }
#if !JVET_N0413_RDPCM
          if( !cu.ispMode )
          {
            bestNormalIntraModeIndex = uiMode;
          }
#endif
#if JVET_N0193_LFNST
          if( !cu.ispMode && !cu.mtsFlag )
          {
            m_modeCtrl->setMtsFirstPassNoIspCost( csBest->cost );
          }
#endif
        }
#if JVET_N0413_RDPCM
        if( !cu.ispMode && !cu.bdpcmMode && csBest->cost < bestCostNonBDPCM )
        {
          bestCostNonBDPCM = csBest->cost;
          bestNormalIntraModeIndex = mode;
        }
#endif
#if JVET_N0193_LFNST
      }
#endif

      csTemp->releaseIntermediateData();
    } // Mode loop//每种模式的循环

	//RD候选模式的循环RDCost遍历结束
#if JVET_N0217_MATRIX_INTRAPRED
    cu.ispMode = uiBestPUMode.ispMod;//将最优的模式用到ISP中
#else
    cu.ispMode = bestIspOption;
#endif

#if JVET_N0193_LFNST
    if( validReturn )
    {
#endif
      cs.useSubStructure( *csBest, partitioner.chType, pu.singleChan( CHANNEL_TYPE_LUMA ), true, true, keepResi, keepResi );
#if JVET_N0193_LFNST
    }
#endif
    csBest->releaseIntermediateData();
#if JVET_N0193_LFNST
    if( validReturn )
    {
#endif
      //=== update PU data ====
		//更新PU的数据
#if JVET_N0217_MATRIX_INTRAPRED
      cu.mipFlag = uiBestPUMode.mipFlg;//cost最优的模式是否是Mip模式
      pu.multiRefIdx = uiBestPUMode.mRefId;//cost最优的模式是否使用多参考行
      pu.intraDir[ CHANNEL_TYPE_LUMA ] = uiBestPUMode.modeId;
#else
      pu.intraDir[ 0 ] = uiBestPUMode;
      pu.multiRefIdx = bestExtendRef;
#endif
#if JVET_N0413_RDPCM
      cu.bdpcmMode = bestBDPCMMode;
#endif
#if JVET_N0193_LFNST
    }
#endif
  }

  //===== reset context models =====
  //重置上下文模型
  m_CABACEstimator->getCtx() = ctxStart;

#if JVET_N0193_LFNST
  return validReturn;//返回是否可用
#endif
}