SRS源码分析--RTMP 视频流转 WebRTC 视频流分析

SrsRtcFromRtmpBridger 类相关方法调用时机

• 创建并初始化 SrsRtcFromRtmpBridger 实例

SrsRtmpConn::publishing(SrsLiveSource* source) ->
SrsRtmpConn::acquire_publish(SrsLiveSource* source) ->
 
寻找或创建一个 RTC 源，即 SrsRtcSource 实例
err = _srs_rtc_sources->fetch_or_create(req, &rtc)
 
创建并初始化 SrsRtcFromRtmpBridger 实例

SrsRtcFromRtmpBridger::on_publish() 函数

调用路径
 
SrsRtmpConn::publishing(SrsLiveSource* source) ->
SrsRtmpConn::acquire_publish(SrsLiveSource* source) ->
SrsLiveSource::on_publish() ->
SrsRtcFromRtmpBridger::on_publish() ->
SrsRtcSource::on_publish()

• SrsRtcFromRtmpBridger::on_video(SrsSharedPtrMessage* msg) 函数

该函数中解析 RTMP 数据包并封装成 WebRTC 支持的 single NALU、STAP-A、FU-A 格式并发送；

调用路径
 
SrsRecvThread::do_cycle() ->
SrsPublishRecvThread::consume(SrsCommonMessage* msg) ->
SrsRtmpConn::handle_publish_message(SrsLiveSource* source, SrsCommonMessage* msg) ->
SrsRtmpConn::process_publish_message(SrsLiveSource* source, SrsCommonMessage* msg) ->
SrsLiveSource::on_video(SrsCommonMessage* shared_video) ->
SrsLiveSource::on_video_imp(SrsSharedPtrMessage* msg) ->
SrsRtcFromRtmpBridger::on_video(SrsSharedPtrMessage* msg)

• 发送 RTP 数据包

• SrsRtcFromRtmpBridger::on_video(SrsSharedPtrMessage* msg) ->
  SrsRtcFromRtmpBridger::consume_packets(vector& pkts) ->
  source_->on_rtp(pkt)，WebRTC 发送 RTP 数据

  代码段 -- 过滤数据

• srs_error_t SrsRtcFromRtmpBridger::on_video(SrsSharedPtrMessage* msg)
  {
   
      ...
   
      bool has_idr = false;
      vector samples;
      if ((err = filter(msg, format, has_idr, samples)) != srs_success) {
          return srs_error_wrap(err, "filter video");
      }
      int nn_samples = (int)samples.size();
   
      ...
   
  }

  本文福利， 免费领取C++音视频学习资料包、技术视频/代码，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，编解码，推拉流，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓

• 关键代码分析 

• SrsSample::parse_bframe() 函数
  • 功能，过滤 B 帧

• H264 码流分层结构

H264 Slice Data 的语法分析

H264 码流 Slice 头部语法分析 

srs_error_t SrsSample::parse_bframe()
{
    srs_error_t err = srs_success;
 
    uint8_t header = bytes[0];
    // 获取 NAL 类型
    SrsAvcNaluType nal_type = (SrsAvcNaluType)(header & kNalTypeMask);
 
    /**
     * SrsAvcNaluTypeNonIDR，非关键帧
     * SrsAvcNaluTypeDataPartitionA，Slice 分片 A
     * SrsAvcNaluTypeIDR，关键帧
     * 
     * 仅有以上的 NAL 单元类型，其 Slice 分片具有头部信息
     */
    if (nal_type != SrsAvcNaluTypeNonIDR && nal_type != SrsAvcNaluTypeDataPartitionA
                                         && nal_type != SrsAvcNaluTypeIDR) {
        return err;
    }
 
    SrsBuffer* stream = new SrsBuffer(bytes, size);
    SrsAutoFree(SrsBuffer, stream);
 
    // Skip nalu header.
    // 跳过 NAL 头部
    stream->skip(1);
 
    SrsBitBuffer bitstream(stream);
    // 分析 slice header 条带头部
    int32_t first_mb_in_slice = 0;
    // 读取 first_mb_in_slice
    if ((err = srs_avc_nalu_read_uev(&bitstream, first_mb_in_slice)) != srs_success) {
        return srs_error_wrap(err, "nalu read uev");
    }
    // 读取 slice_type
    int32_t slice_type_v = 0;
    if ((err = srs_avc_nalu_read_uev(&bitstream, slice_type_v)) != srs_success) {
        return srs_error_wrap(err, "nalu read uev");
    }
    SrsAvcSliceType slice_type = (SrsAvcSliceType)slice_type_v;
    // 判断条带类型是否为 B 帧类型
    if (slice_type == SrsAvcSliceTypeB || slice_type == SrsAvcSliceTypeB1) {
        bframe = true;
        srs_verbose("nal_type=%d, slice type=%d", nal_type, slice_type);
    }
 
    return err;
}

• srs_avc_nalu_read_uev(SrsBitBuffer* stream, int32_t& v)
  • 功能，解析哥伦布码，即 H264 中的 me(v)、se(v)、te(v)、ue(v)

me(v)、se(v)、te(v)、ue(v) 的含义

me(v): mapped Exp-Golomb-coded syntax element with the left bit first
       映射的指数哥伦布码编码语法元素，左位在先
se(v): signed integer Exp-Golomb-coded syntax element with the left bit first
       有符号的指数哥伦布码编码语法元素，左位在先
te(v): truncated Exp-Golomb-coded syntax element with left bit first
       舍位指数哥伦布码编码语法元素，左位在先
ue(v): unsigned integer Exp-Golomb-coded syntax element with the left bit first
       无符号的指数哥伦布码编码语法元素，左位在先

哥伦布码解码算法

示例，010 解码为 2^1-1+0=1，00110 解码为 2^2-1+2=5

srs_error_t srs_avc_nalu_read_uev(SrsBitBuffer* stream, int32_t& v)
{
    srs_error_t err = srs_success;
    
    if (stream->empty()) {
        return srs_error_new(ERROR_AVC_NALU_UEV, "empty stream");
    }
    
    // ue(v) in 9.1 Parsing process for Exp-Golomb codes
    // ISO_IEC_14496-10-AVC-2012.pdf, page 227.
    // Syntax elements coded as ue(v), me(v), or se(v) are Exp-Golomb-coded.
    //      leadingZeroBits = -1;
    //      for( b = 0; !b; leadingZeroBits++ )
    //          b = read_bits( 1 )
    // The variable codeNum is then assigned as follows:
    //      codeNum = (2<empty(); leadingZeroBits++) {
        b = stream->read_bit();
    }
    
    if (leadingZeroBits >= 31) {
        return srs_error_new(ERROR_AVC_NALU_UEV, "%dbits overflow 31bits", leadingZeroBits);
    }
    
    v = (1 << leadingZeroBits) - 1;
    for (int i = 0; i < (int)leadingZeroBits; i++) {
        if (stream->empty()) {
            return srs_error_new(ERROR_AVC_NALU_UEV, "no bytes for leadingZeroBits=%d", leadingZeroBits);
        }
        
        int32_t b = stream->read_bit();
        v += b << (leadingZeroBits - 1 - i);
    }
    
    return err;
}

本文福利， 免费领取C++音视频学习资料包、技术视频/代码，内容包括（音视频开发，面试题，FFmpeg ，webRTC ，rtmp ，hls ，rtsp ，ffplay ，编解码，推拉流，srs）↓↓↓↓↓↓见下面↓↓文章底部点击免费领取↓↓