音视频从入门到精通——FFmpeg结构体：AVStream分析

FFmpeg结构体：AVStream分析

概述

AVStream是存储每一个视频/音频流信息的结构体。本文分析一下该结构体里重要变量的含义和作用。该结构体位于avformat.h中，首先看一下结构体的定义：

代码

/**
 * Stream structure.
 * New fields can be added to the end with minor version bumps.
 * Removal, reordering and changes to existing fields require a major
 * version bump.
 * sizeof(AVStream) must not be used outside libav*.
 */
typedef struct AVStream {
#if FF_API_AVSTREAM_CLASS
    /**
     * A class for @ref avoptions. Set on stream creation.
     */
    const AVClass *av_class;
#endif

    int index;    /**< stream index in AVFormatContext */
    /**
     * Format-specific stream ID.
     * decoding: set by libavformat
     * encoding: set by the user, replaced by libavformat if left unset
     */
    int id;

    void *priv_data;

    /**
     * This is the fundamental unit of time (in seconds) in terms
     * of which frame timestamps are represented.
     *
     * decoding: set by libavformat
     * encoding: May be set by the caller before avformat_write_header() to
     *           provide a hint to the muxer about the desired timebase. In
     *           avformat_write_header(), the muxer will overwrite this field
     *           with the timebase that will actually be used for the timestamps
     *           written into the file (which may or may not be related to the
     *           user-provided one, depending on the format).
     */
    AVRational time_base;

    /**
     * Decoding: pts of the first frame of the stream in presentation order, in stream time base.
     * Only set this if you are absolutely 100% sure that the value you set
     * it to really is the pts of the first frame.
     * This may be undefined (AV_NOPTS_VALUE).
     * @note The ASF header does NOT contain a correct start_time the ASF
     * demuxer must NOT set this.
     */
    int64_t start_time;

    /**
     * Decoding: duration of the stream, in stream time base.
     * If a source file does not specify a duration, but does specify
     * a bitrate, this value will be estimated from bitrate and file size.
     *
     * Encoding: May be set by the caller before avformat_write_header() to
     * provide a hint to the muxer about the estimated duration.
     */
    int64_t duration;

    int64_t nb_frames;                 ///< number of frames in this stream if known or 0

    /**
     * Stream disposition - a combination of AV_DISPOSITION_* flags.
     * - demuxing: set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: may be set by the caller before avformat_write_header().
     */
    int disposition;

    enum AVDiscard discard; ///< Selects which packets can be discarded at will and do not need to be demuxed.

    /**
     * sample aspect ratio (0 if unknown)
     * - encoding: Set by user.
     * - decoding: Set by libavformat.
     */
    AVRational sample_aspect_ratio;

    AVDictionary *metadata;

    /**
     * Average framerate
     *
     * - demuxing: May be set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: May be set by the caller before avformat_write_header().
     */
    AVRational avg_frame_rate;

    /**
     * For streams with AV_DISPOSITION_ATTACHED_PIC disposition, this packet
     * will contain the attached picture.
     *
     * decoding: set by libavformat, must not be modified by the caller.
     * encoding: unused
     */
    AVPacket attached_pic;

    /**
     * An array of side data that applies to the whole stream (i.e. the
     * container does not allow it to change between packets).
     *
     * There may be no overlap between the side data in this array and side data
     * in the packets. I.e. a given side data is either exported by the muxer
     * (demuxing) / set by the caller (muxing) in this array, then it never
     * appears in the packets, or the side data is exported / sent through
     * the packets (always in the first packet where the value becomes known or
     * changes), then it does not appear in this array.
     *
     * - demuxing: Set by libavformat when the stream is created.
     * - muxing: May be set by the caller before avformat_write_header().
     *
     * Freed by libavformat in avformat_free_context().
     *
     * @see av_format_inject_global_side_data()
     */
    AVPacketSideData *side_data;
    /**
     * The number of elements in the AVStream.side_data array.
     */
    int            nb_side_data;

    /**
     * Flags indicating events happening on the stream, a combination of
     * AVSTREAM_EVENT_FLAG_*.
     *
     * - demuxing: may be set by the demuxer in avformat_open_input(),
     *   avformat_find_stream_info() and av_read_frame(). Flags must be cleared
     *   by the user once the event has been handled.
     * - muxing: may be set by the user after avformat_write_header(). to
     *   indicate a user-triggered event.  The muxer will clear the flags for
     *   events it has handled in av_[interleaved]_write_frame().
     */
    int event_flags;
/**
 * - demuxing: the demuxer read new metadata from the file and updated
 *     AVStream.metadata accordingly
 * - muxing: the user updated AVStream.metadata and wishes the muxer to write
 *     it into the file
 */
#define AVSTREAM_EVENT_FLAG_METADATA_UPDATED 0x0001
/**
 * - demuxing: new packets for this stream were read from the file. This
 *   event is informational only and does not guarantee that new packets
 *   for this stream will necessarily be returned from av_read_frame().
 */
#define AVSTREAM_EVENT_FLAG_NEW_PACKETS (1 << 1)

    /**
     * Real base framerate of the stream.
     * This is the lowest framerate with which all timestamps can be
     * represented accurately (it is the least common multiple of all
     * framerates in the stream). Note, this value is just a guess!
     * For example, if the time base is 1/90000 and all frames have either
     * approximately 3600 or 1800 timer ticks, then r_frame_rate will be 50/1.
     */
    AVRational r_frame_rate;

    /**
     * Codec parameters associated with this stream. Allocated and freed by
     * libavformat in avformat_new_stream() and avformat_free_context()
     * respectively.
     *
     * - demuxing: filled by libavformat on stream creation or in
     *             avformat_find_stream_info()
     * - muxing: filled by the caller before avformat_write_header()
     */
    AVCodecParameters *codecpar;

    /**
     * Number of bits in timestamps. Used for wrapping control.
     *
     * - demuxing: set by libavformat
     * - muxing: set by libavformat
     *
     */
    int pts_wrap_bits;
} AVStream;

分析

AVStream 主要成员

我们来分析几个参数看看

第一个，index

int index;    /**< stream index in AVFormatContext */

AVFormatContext中的流索引，用来标识是视频还是音频 一般0是视频，1是音频。实际上没什么用。
判断流是视频还是音频通过，AVMEDIA_TYPE_AUDIO音频，AVMEDIA_TYPE_VIDEO视频来判断。

第二个，time_base

    /**
     * This is the fundamental unit of time (in seconds) in terms
     * of which frame timestamps are represented.
     * 这是表示帧时间戳的基本时间单位（秒）。
     *
     * decoding: set by libavformat
     * encoding: May be set by the caller before avformat_write_header() to
     *           provide a hint to the muxer about the desired timebase. In
     *           avformat_write_header(), the muxer will overwrite this field
     *           with the timebase that will actually be used for the timestamps
     *           written into the file (which may or may not be related to the
     *           user-provided one, depending on the format).
     */
    AVRational time_base;

看这个AVRational结构体，它在AVStream内大量使用。
有理数是整数和分数的集合 ; 有理数可以用两个整数相除 ( 分数 ) 来表示。所以，AVRational是一个有理数。

/**
 * @defgroup lavu_math_rational AVRational
 * @ingroup lavu_math
 * Rational number calculation.
 *
 * While rational numbers can be expressed as floating-point numbers, the
 * conversion process is a lossy one, so are floating-point operations. On the
 * other hand, the nature of FFmpeg demands highly accurate calculation of
 * timestamps. This set of rational number utilities serves as a generic
 * interface for manipulating rational numbers as pairs of numerators and
 * denominators.
 *
 * Many of the functions that operate on AVRational's have the suffix `_q`, in
 * reference to the mathematical symbol "ℚ" (Q) which denotes the set of all
 * rational numbers.
 *
 */

/**
 * Rational number (pair of numerator and denominator).
 */
typedef struct AVRational{
    int num; ///< Numerator---分子
    int den; ///< Denominator---分母
} AVRational;

所以，AVStream中的time_base 是一个分数，它比float，double类似更加精确。

第三个，duration

/**
 * Decoding: duration of the stream, in stream time base.
 * If a source file does not specify a duration, but does specify
 * a bitrate, this value will be estimated from bitrate and file size.
 *
 * Encoding: May be set by the caller before avformat_write_header() to
 * provide a hint to the muxer about the estimated duration.
 */
解码：流的持续时间，流时基。
如果源文件未指定持续时间，但指定
比特率，该值将根据比特率和文件大小估计。

编码：可以由调用方在avformat_write_header（）之前设置为
向多路复用器提供有关估计持续时间的提示。

    int64_t  duration;

流的持续时间，这个数的值不是秒，而是一个以time base为基准的统计上的数量。大概是可以通过这个数算出这个视频，或者音频的时长。
计算方法如下

视频总时长(毫秒) 
duration * ( (double)time_base.num / (double)time_base.den ) * 1000

视频总时长(秒) 
duration * ( (double)time_base.num / (double)time_base.den ) 

注意，time_base.den有可能为0，这里需要写一个函数来判断。

第四个，codec

AVCodecContext *codec ，指向该视频/音频流的AVCodecContext（它们是一一对应的关系）。
这个参数在AVStream代码里没有找到，它已经被第五个参数codecpar代替了。

第五个，codecpar

    /**
     * Codec parameters associated with this stream. Allocated and freed by
     * libavformat in avformat_new_stream() and avformat_free_context()
     * respectively.
     *
     * - demuxing: filled by libavformat on stream creation or in
     *             avformat_find_stream_info()
     * - muxing: filled by the caller before avformat_write_header()
     */
    AVCodecParameters *codecpar;

音视频参数，这个参数是用来替代上面的codec的。

因为AVCodecContext结构体包含的参数太多，AVCodecParameters将编码器的参数从AVCodecContext分离出来，AVCodecParameters结构体中部分重要的参数如下：

enum AVMediaType codec_type; 编解码器的类型 
const struct AVCodec *codec; 编解码器,初始化后不可更改
enum AVCodecID codec_id; 编解码器的id
int64_t bit_rate; 平均比特率
uint8_t *extradata; int extradata_size; 针对特定编码器包含的附加信息
AVRational time_base; 根据该参数可以将pts转化为实践
int width, height; 每一帧的宽和高
int gop_size; 一组图片的数量，编码时用户设置，解码时不使用
enum AVPixelFormat pix_fmt; 像素格式，编码时用户设置，解码时可由用户指定，但是在分析数据会被覆盖用户的设置
int refs; 参考帧的数量
enum AVColorSpace colorspace; YUV色彩空间类型
enum AVColorRange color_range; MPEG JPEG YUV范围
int sample_rate; 采样率 仅音频
int channels; 声道数（音频）
enum AVSampleFormat sample_fmt; //采样格式
int frame_size; 每个音频帧中每个声道的采样数量
int profile;配置类型
int level;级别

第六个,avg_frame_rate

avg_frame_rate表示，平均帧率。

    /**
     * Average framerate
     *
     * - demuxing: May be set by libavformat when creating the stream or in
     *             avformat_find_stream_info().
     * - muxing: May be set by the caller before avformat_write_header().
     */
    AVRational avg_frame_rate;

帧率 ( FPS ) : 单位时间内 ( 1 秒 ) , 需要显示的图像个数 , 单位是 Hz ;
计算方法

//获取视频的 FPS 帧率 ( 1秒中播放的帧数 )

AVRational frame_rate = stream->avg_frame_rate;

// AVRational 结构体由一个分子和分母组成 , 分子 / 分母就是 fps
//  也可以使用 av_q2d() 方法传入 AVRational 结构体进行计算
//  上面两种方法都可以获取 帧率 ( FPS )
//      FPS 的值不是固定的 , 随着视频播放 , 其帧率也会随之改变
int fps = frame_rate.num / frame_rate.den;
//int fps = av_q2d(frame_rate);

如果 FPS 为 100Hz , 那么1 秒钟绘制 100 张画面 , 每隔 10ms 绘制一张图像。

第七个，attached_pic

附带的图片。比如说一些MP3，AAC音频文件附带的专辑封面。

    /**
     * For streams with AV_DISPOSITION_ATTACHED_PIC disposition, this packet
     * will contain the attached picture.
     *
     * decoding: set by libavformat, must not be modified by the caller.
     * encoding: unused
     */
    AVPacket attached_pic;

其他成员

看附录

附录

int index; 标识该音频/视频流
int id; 流的标识，依赖于具体的容器格式。解码libavformat,编码由用户设置，用户不设置则使用libavformat
AVCodecContext *codec; 流对应与AVCodecContext结构，调用avformat_open_input设置
AVRational time_base; 表示帧时间戳的基本单位。通过该值可以把PTS,DTS转化为真正的时间，其他结构体中也有这个字段，只有AVStream 中的time_base为真正的时间。
int64_t start_time; 流的起始时间，以流的基准时间为单位
int64_t duration; 流的持续时间，如果源文件未指定持续时间，但指定了比特率，则将根据比特率和文件大小估计该值。
int64_t nb_frames; 流中帧的数据，为0或者已知
AVDictionary *metadata; 元数据
AVRational sample_aspect_ratio; 样本长宽比
AVRational avg_frame_rate; 评价帧率，解封装时：在创建流时设置或者在avformat_find_stream_info()中设置；封装时调用avformat_write_header()设置
AVCodecParameters *codecpar; 编解码器参数
int64_t first_dts; 第一个dts
int64_t cur_dts; 当前dts
int probe_packets; 编码器用户probe的包的个数
int codec_info_nb_frames; 在avformat_find_stream_info()期间已经解封装的帧数
AVPacket attached_pic;附带的一些图片
int request_probe; 流探测状态，-1表示完成，0表示没有探测请求，rest执行探测
int skip_to_keyframe; 是否丢弃所有内容直接跳到关键帧
int skip_samples;在下一个数据表解码的帧开始要跳过的采样数
int64_t start_skip_samples;从流的开始要跳过的采样的数量
int64_t first_discard_sample; 如果不是0，从流中丢弃第一个音频的样本
int nb_decoded_frames; 内部解码的帧数

int64_t pts_reorder_error[MAX_REORDER_DELAY+1];
uint8_t pts_reorder_error_count[MAX_REORDER_DELAY+1]; 内部数据从pts到dts

int64_t last_dts_for_order_check; 内部数据用于分析dts和探测mpeg流的错误
uint8_t dts_ordered;
uint8_t dts_misordered;

AVRational display_aspect_ratio; 显示的宽高比

参考

FFmpeg结构体：AVStream
雷霄骅FFMPEG结构体分析：AVStream
某乎-FFMPEG结构体：AVStream
Android FFMPEG 开发FFMPEG 音视频同步