webrtc 视频编码之 h264 自动调节分辨率二

上次说到 openh264 的编码分辨率是通过 QualityScaler 计算得来的，继续看

看代码实现

初始化接口

void QualityScaler::Init(int low_qp_threshold,
                         int high_qp_threshold,
                         int initial_bitrate_kbps,
                         int width,
                         int height,
                         int fps) {
  ClearSamples();
  low_qp_threshold_ = low_qp_threshold;
  high_qp_threshold_ = high_qp_threshold;
  downscale_shift_ = 0;
  fast_rampup_ = true;

  const int init_width = width;
  const int init_height = height;
  if (initial_bitrate_kbps > 0) {
    int init_num_pixels = width * height;
    if (initial_bitrate_kbps < kVgaBitrateThresholdKbps)
      init_num_pixels = kVgaNumPixels;
    if (initial_bitrate_kbps < kQvgaBitrateThresholdKbps)
      init_num_pixels = kQvgaNumPixels;
    while (width * height > init_num_pixels) {
      ++downscale_shift_;
      width /= 2;
      height /= 2;
    }
  }
  UpdateTargetResolution(init_width, init_height);
  ReportFramerate(fps);
}

编码器对不同的编码分辨率预设了码率范围，当码率超出对应的范围时，会去调整分辨率

对应范围在

static const int kVgaBitrateThresholdKbps = 500;
static const int kVgaNumPixels = 700 * 500;  // 640x480
static const int kQvgaBitrateThresholdKbps = 250;
static const int kQvgaNumPixels = 400 * 300;  // 320x240

视频清晰度上报

void QualityScaler::ReportQP(int qp) {
  framedrop_percent_.AddSample(0);
  average_qp_.AddSample(qp);
}

qp 表明了当前帧的清晰度，越大视频越模糊，越小越清晰（0-51），当编码器编完时，会计算当前帧的qp，并上报

编码器丢帧上报

void QualityScaler::ReportDroppedFrame() {
  framedrop_percent_.AddSample(100);
}

当码率太小或者画面剧烈运动时，编码需要的码率增大，但是设置的目标码率太小，导致编码失败，此时上报丢帧

编码器编码前对原始图像拉伸计算

void QualityScaler::OnEncodeFrame(int width, int height) {
  // Should be set through InitEncode -> Should be set by now.
  RTC_DCHECK_GE(low_qp_threshold_, 0);
  if (target_res_.width != width || target_res_.height != height) {
    UpdateTargetResolution(width, height);
  }

  // Check if we should scale down due to high frame drop.
  const auto drop_rate = framedrop_percent_.GetAverage(num_samples_downscale_);
  if (drop_rate && *drop_rate >= kFramedropPercentThreshold) {
    ScaleDown();
    return;
  }

  // Check if we should scale up or down based on QP.
  const auto avg_qp_down = average_qp_.GetAverage(num_samples_downscale_);
  if (avg_qp_down && *avg_qp_down > high_qp_threshold_) {
    ScaleDown();
    return;
  }
  const auto avg_qp_up = average_qp_.GetAverage(num_samples_upscale_);
  if (avg_qp_up && *avg_qp_up <= low_qp_threshold_) {
    // QP has been low. We want to try a higher resolution.
    ScaleUp();
    return;
  }
}

首先是按照码率计算的来一下调整 再根据丢帧率百分比计算

yuv 图像拉伸

rtc::scoped_refptr QualityScaler::GetScaledBuffer(
    const rtc::scoped_refptr& frame)

参数设定

static const int kMeasureSecondsUpscale = 5; //调整时间间隔 避免频繁调整 导致画面一直变化
static const int kMeasureSecondsDownscale = 5;
static const int kFramedropPercentThreshold = 60; //编码器丢帧率
// Min width/height to downscale to, set to not go below QVGA, but with some
// margin to permit "almost-QVGA" resolutions, such as QCIF.
static const int kMinDownscaleDimension = 140;  //最小分辨率

static const int kLowH264QpThreshold = 24; //最小qp 小于此值 会提高分辨率
static const int kHighH264QpThreshold = 37; //最大qp 大于此值 会降低分辨率

ok，我们可以修改这些参数 设置最小的分辨率，调整时间等的