H264 RTP与RTCP SR

1. 网络抽象层单元类型 (NALU)

NALU 头由一个字节组成, 它的语法如下:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+

F: 1 个比特.
  forbidden_zero_bit. 在 H.264 规范中规定了这一位必须为 0.

NRI: 2 个比特.
  nal_ref_idc. 取 00 ~ 11, 似乎指示这个 NALU 的重要性, 如 00 的 NALU 解码器可以丢弃它而不影响图像的回放. 不过一般情况下不太关心

这个属性.

Type: 5 个比特.
  nal_unit_type. 这个 NALU 单元的类型. 简述如下:

  0     没有定义
  1-23  NAL单元  单个 NAL 单元包.
  24    STAP-A   单一时间的组合包
  24    STAP-B   单一时间的组合包
  26    MTAP16   多个时间的组合包
  27    MTAP24   多个时间的组合包
  28    FU-A     分片的单元
  29    FU-B     分片的单元
  30-31 没有定义

2. 打包模式

  下面是 RFC 3550 中规定的 RTP 头的结构.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |V=2|P|X|  CC   |M|     PT      |       sequence number         |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                           timestamp                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           synchronization source (SSRC) identifier            |
      +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
      |            contributing source (CSRC) identifiers             |
      |                             ....                              |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

  负载类型 Payload type (PT): 7 bits
  序列号 Sequence number (SN): 16 bits
  时间戳 Timestamp: 32 bits
  
  H.264 Payload 格式定义了三种不同的基本的负载(Payload)结构. 接收端可能通过 RTP Payload 
  的第一个字节来识别它们. 这一个字节类似 NALU 头的格式, 而这个头结构的 NAL 单元类型字段
  则指出了代表的是哪一种结构,

  这个字节的结构如下, 可以看出它和 H.264 的 NALU 头结构是一样的.
      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+
  字段 Type: 这个 RTP payload 中 NAL 单元的类型. 这个字段和 H.264 中类型字段的区别是, 当 type
  的值为 24 ~ 31 表示这是一个特别格式的 NAL 单元, 而 H.264 中, 只取 1~23 是有效的值.
   
  24    STAP-A   单一时间的组合包
  24    STAP-B   单一时间的组合包
  26    MTAP16   多个时间的组合包
  27    MTAP24   多个时间的组合包
  28    FU-A     分片的单元
  29    FU-B     分片的单元
  30-31 没有定义

  可能的结构类型分别有:

  1. 单一 NAL 单元模式
     即一个 RTP 包仅由一个完整的 NALU 组成. 这种情况下 RTP NAL 头类型字段和原始的 H.264的
  NALU 头类型字段是一样的.

  2. 组合封包模式
    即可能是由多个 NAL 单元组成一个 RTP 包. 分别有4种组合方式: STAP-A, STAP-B, MTAP16, MTAP24.
  那么这里的类型值分别是 24, 25, 26 以及 27.

  3. 分片封包模式
    用于把一个 NALU 单元封装成多个 RTP 包. 存在两种类型 FU-A 和 FU-B. 类型值分别是 28 和 29.

2.1 单一 NAL 单元模式

  对于 NALU 的长度小于 MTU 大小的包, 一般采用单一 NAL 单元模式.
  对于一个原始的 H.264 NALU 单元常由 [Start Code] [NALU Header] [NALU Payload] 三部分组成, 其中 Start Code 用于标示这是一个

NALU 单元的开始, 必须是 "00 00 00 01" 或 "00 00 01", NALU 头仅一个字节, 其后都是 NALU 单元内容.
  打包时去除 "00 00 01" 或 "00 00 00 01" 的开始码, 把其他数据封包的 RTP 包即可.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |F|NRI|  type   |                                               |
      +-+-+-+-+-+-+-+-+                                               |
      |                                                               |
      |               Bytes 2..n of a Single NAL unit                 |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


  如有一个 H.264 的 NALU 是这样的:

  [00 00 00 01 67 42 A0 1E 23 56 0E 2F ... ]

  这是一个序列参数集 NAL 单元. [00 00 00 01] 是四个字节的开始码, 67 是 NALU 头, 42 开始的数据是 NALU 内容.

  封装成 RTP 包将如下:

  [ RTP Header ] [ 67 42 A0 1E 23 56 0E 2F ]

  即只要去掉 4 个字节的开始码就可以了.


2.2 组合封包模式

  其次, 当 NALU 的长度特别小时, 可以把几个 NALU 单元封在一个 RTP 包中.


       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                          RTP Header                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |STAP-A NAL HDR |         NALU 1 Size           | NALU 1 HDR    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         NALU 1 Data                           |
      :                                                               :
      +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               | NALU 2 Size                   | NALU 2 HDR    |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         NALU 2 Data                           |
      :                                                               :
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+


2.3 Fragmentation Units (FUs).

  而当 NALU 的长度超过 MTU 时, 就必须对 NALU 单元进行分片封包. 也称为 Fragmentation Units (FUs).
  
       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      | FU indicator  |   FU header   |                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               |
      |                                                               |
      |                         FU payload                            |
      |                                                               |
      |                               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                               :...OPTIONAL RTP padding        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 14.  RTP payload format for FU-A

   The FU indicator octet has the following format:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |F|NRI|  Type   |
      +---------------+

   The FU header has the following format:

      +---------------+
      |0|1|2|3|4|5|6|7|
      +-+-+-+-+-+-+-+-+
      |S|E|R|  Type   |
      +---------------+

 

SR: Sender Report RTCP Packet

 
        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
header |V=2|P|    RC   |   PT=SR=200   |             length            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                         SSRC of sender                        |
       +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
sender |              NTP timestamp, most significant word             |
info   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |             NTP timestamp, least significant word             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                         RTP timestamp                         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                     sender's packet count                     |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                      sender's octet count                     |
       +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
report |                 SSRC_1 (SSRC of first source)                 |
block  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  1    | fraction lost |       cumulative number of packets lost       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |           extended highest sequence number received           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                      interarrival jitter                      |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                         last SR (LSR)                         |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                   delay since last SR (DLSR)                  |
       +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
report |                 SSRC_2 (SSRC of second source)                |
block  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
  2    :                               ...                             :
       +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
       |                  profile-specific extensions                  |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
 
   The sender report packet consists of three sections, possibly
   followed by a fourth profile-specific extension section if defined.
   The first section, the header, is 8 octets long.  The fields have the
   following meaning:
 
   version (V): 2 bits
      Identifies the version of RTP, which is the same in RTCP packets
      as in RTP data packets.  The version defined by this specification
      is two (2).
 
   padding (P): 1 bit
      If the padding bit is set, this individual RTCP packet contains
      some additional padding octets at the end which are not part of
      the control information but are included in the length field.  The
      last octet of the padding is a count of how many padding octets
      should be ignored, including itself (it will be a multiple of
      four).  Padding may be needed by some encryption algorithms with
      fixed block sizes.  In a compound RTCP packet, padding is only
      required on one individual packet because the compound packet is
      encrypted as a whole for the method in Section 9.1.  Thus, padding
      MUST only be added to the last individual packet, and if padding
      is added to that packet, the padding bit MUST be set only on that
      packet.  This convention aids the header validity checks described
      in Appendix A.2 and allows detection of packets from some early
      implementations that incorrectly set the padding bit on the first
      individual packet and add padding to the last individual packet.
 
   reception report count (RC): 5 bits
      The number of reception report blocks contained in this packet.  A
      value of zero is valid.
 
   packet type (PT): 8 bits
      Contains the constant 200 to identify this as an RTCP SR packet.
 
   length: 16 bits
      The length of this RTCP packet in 32-bit words minus one,
      including the header and any padding.  (The offset of one makes
      zero a valid length and avoids a possible infinite loop in
      scanning a compound RTCP packet, while counting 32-bit words
      avoids a validity check for a multiple of 4.)
 
   SSRC: 32 bits
      The synchronization source identifier for the originator of this
      SR packet.
 
   The second section, the sender information, is 20 octets long and is
   present in every sender report packet.  It summarizes the data
   transmissions from this sender.  The fields have the following
   meaning:
 
   NTP timestamp: 64 bits
      Indicates the wallclock time (see Section 4) when this report was
      sent so that it may be used in combination with timestamps
      returned in reception reports from other receivers to measure
      round-trip propagation to those receivers.  Receivers should
      expect that the measurement accuracy of the timestamp may be
      limited to far less than the resolution of the NTP timestamp.  The
      measurement uncertainty of the timestamp is not indicated as it
 
      may not be known.  On a system that has no notion of wallclock
      time but does have some system-specific clock such as "system
      uptime", a sender MAY use that clock as a reference to calculate
      relative NTP timestamps.  It is important to choose a commonly
      used clock so that if separate implementations are used to produce
      the individual streams of a multimedia session, all
      implementations will use the same clock.  Until the year 2036,
      relative and absolute timestamps will differ in the high bit so
      (invalid) comparisons will show a large difference; by then one
      hopes relative timestamps will no longer be needed.  A sender that
      has no notion of wallclock or elapsed time MAY set the NTP
      timestamp to zero.
 
   RTP timestamp: 32 bits
      Corresponds to the same time as the NTP timestamp (above), but in
      the same units and with the same random offset as the RTP
      timestamps in data packets.  This correspondence may be used for
      intra- and inter-media synchronization for sources whose NTP
      timestamps are synchronized, and may be used by media-independent
      receivers to estimate the nominal RTP clock frequency.  Note that
      in most cases this timestamp will not be equal to the RTP
      timestamp in any adjacent data packet.  Rather, it MUST be
      calculated from the corresponding NTP timestamp using the
      relationship between the RTP timestamp counter and real time as
      maintained by periodically checking the wallclock time at a
      sampling instant.
 
   sender's packet count: 32 bits
      The total number of RTP data packets transmitted by the sender
      since starting transmission up until the time this SR packet was
      generated.  The count SHOULD be reset if the sender changes its
      SSRC identifier.
 
   sender's octet count: 32 bits
      The total number of payload octets (i.e., not including header or
      padding) transmitted in RTP data packets by the sender since
      starting transmission up until the time this SR packet was
      generated.  The count SHOULD be reset if the sender changes its
      SSRC identifier.  This field can be used to estimate the average
      payload data rate.
 
   The third section contains zero or more reception report blocks
   depending on the number of other sources heard by this sender since
   the last report.  Each reception report block conveys statistics on
   the reception of RTP packets from a single synchronization source.
   Receivers SHOULD NOT carry over statistics when a source changes its
   SSRC identifier due to a collision.  These statistics are:
 
   SSRC_n (source identifier): 32 bits
      The SSRC identifier of the source to which the information in this
      reception report block pertains.
 
   fraction lost: 8 bits
      The fraction of RTP data packets from source SSRC_n lost since the
      previous SR or RR packet was sent, expressed as a fixed point
      number with the binary point at the left edge of the field.  (That
      is equivalent to taking the integer part after multiplying the
      loss fraction by 256.)  This fraction is defined to be the number
      of packets lost divided by the number of packets expected, as
      defined in the next paragraph.  An implementation is shown in
      Appendix A.3.  If the loss is negative due to duplicates, the
      fraction lost is set to zero.  Note that a receiver cannot tell
      whether any packets were lost after the last one received, and
      that there will be no reception report block issued for a source
      if all packets from that source sent during the last reporting
      interval have been lost.
 
   cumulative number of packets lost: 24 bits
      The total number of RTP data packets from source SSRC_n that have
      been lost since the beginning of reception.  This number is
      defined to be the number of packets expected less the number of
      packets actually received, where the number of packets received
      includes any which are late or duplicates.  Thus, packets that
      arrive late are not counted as lost, and the loss may be negative
      if there are duplicates.  The number of packets expected is
      defined to be the extended last sequence number received, as
      defined next, less the initial sequence number received.  This may
      be calculated as shown in Appendix A.3.
 
   extended highest sequence number received: 32 bits
      The low 16 bits contain the highest sequence number received in an
      RTP data packet from source SSRC_n, and the most significant 16
      bits extend that sequence number with the corresponding count of
      sequence number cycles, which may be maintained according to the
      algorithm in Appendix A.1.  Note that different receivers within
      the same session will generate different extensions to the
      sequence number if their start times differ significantly.
 
   interarrival jitter: 32 bits
      An estimate of the statistical variance of the RTP data packet
      interarrival time, measured in timestamp units and expressed as an
      unsigned integer.  The interarrival jitter J is defined to be the
      mean deviation (smoothed absolute value) of the difference D in
      packet spacing at the receiver compared to the sender for a pair
      of packets.  As shown in the equation below, this is equivalent to
      the difference in the "relative transit time" for the two packets;
 
      the relative transit time is the difference between a packet's RTP
      timestamp and the receiver's clock at the time of arrival,
      measured in the same units.
 
      If Si is the RTP timestamp from packet i, and Ri is the time of
      arrival in RTP timestamp units for packet i, then for two packets
      i and j, D may be expressed as
 
         D(i,j) = (Rj - Ri) - (Sj - Si) = (Rj - Sj) - (Ri - Si)
 
      The interarrival jitter SHOULD be calculated continuously as each
      data packet i is received from source SSRC_n, using this
      difference D for that packet and the previous packet i-1 in order
      of arrival (not necessarily in sequence), according to the formula
 
         J(i) = J(i-1) + (|D(i-1,i)| - J(i-1))/16
 
      Whenever a reception report is issued, the current value of J is
      sampled.
 
      The jitter calculation MUST conform to the formula specified here
      in order to allow profile-independent monitors to make valid
      interpretations of reports coming from different implementations.
      This algorithm is the optimal first-order estimator and the gain
      parameter 1/16 gives a good noise reduction ratio while
      maintaining a reasonable rate of convergence [22].  A sample
      implementation is shown in Appendix A.8.  See Section 6.4.4 for a
      discussion of the effects of varying packet duration and delay
      before transmission.
 
   last SR timestamp (LSR): 32 bits
      The middle 32 bits out of 64 in the NTP timestamp (as explained in
      Section 4) received as part of the most recent RTCP sender report
      (SR) packet from source SSRC_n.  If no SR has been received yet,
      the field is set to zero.
 
   delay since last SR (DLSR): 32 bits
      The delay, expressed in units of 1/65536 seconds, between
      receiving the last SR packet from source SSRC_n and sending this
      reception report block.  If no SR packet has been received yet
      from SSRC_n, the DLSR field is set to zero.
 
      Let SSRC_r denote the receiver issuing this receiver report.
      Source SSRC_n can compute the round-trip propagation delay to
      SSRC_r by recording the time A when this reception report block is
      received.  It calculates the total round-trip time A-LSR using the
      last SR timestamp (LSR) field, and then subtracting this field to
      leave the round-trip propagation delay as (A - LSR - DLSR).  This
 
      is illustrated in Fig. 2.  Times are shown in both a hexadecimal
      representation of the 32-bit fields and the equivalent floating-
      point decimal representation.  Colons indicate a 32-bit field
      divided into a 16-bit integer part and 16-bit fraction part.
 
      This may be used as an approximate measure of distance to cluster
      receivers, although some links have very asymmetric delays.
 
   [10 Nov 1995 11:33:25.125 UTC]       [10 Nov 1995 11:33:36.5 UTC]
   n                 SR(n)              A=b710:8000 (46864.500 s)
   ---------------------------------------------------------------->
                      v                 ^
   ntp_sec =0xb44db705 v               ^ dlsr=0x0005:4000 (    5.250s)
   ntp_frac=0x20000000  v             ^  lsr =0xb705:2000 (46853.125s)
     (3024992005.125 s)  v           ^
   r                      v         ^ RR(n)
   ---------------------------------------------------------------->
                          |<-DLSR->|
                           (5.250 s)
 
   A     0xb710:8000 (46864.500 s)
   DLSR -0x0005:4000 (    5.250 s)
   LSR  -0xb705:2000 (46853.125 s)
   -------------------------------
   delay 0x0006:2000 (    6.125 s)
H264 RTP与RTCP SR_第1张图片

你可能感兴趣的:(H264 RTP与RTCP SR)