ffmpeg下HLS解析过程

三篇相关联的文章： 

ffmpeg下HLS解析过程-CSDN博客
TS文件格式详解及解封装过程-CSDN博客

FFMPEG解析ts流-CSDN博客

简介

       HLS (HTTP Live Streaming)是Apple的动态码率自适应技术。主要用于PC和Apple终端的音视频服务。包括一个m3u(8)的索引文件，TS媒体分片文件和key加密串文件。

        它的工作原理是将一个大的媒体文件进行分片，将该分片文件资源路径记录于 m3u8 文件（即 playlist）内。当媒体流正在播放时，客户端可以选择从许多不同的备用源中以不同的速率下载同样的资源，允许流媒体会话适应不同的数据速率。在开始一个流媒体会话时，客户端会下载一个包含元数据的extended M3U (m3u8) playlist文件，用于寻找可用的媒体流。
        HLS只请求基本的HTTP报文，与实时传输协议（RTP）不同，HLS可以穿过任何允许HTTP数据通过的防火墙或者代理服务器。它也很容易使用内容分发网络来传输媒体流。

        本文重点介绍HLS相关的解析过程，有关HTTP的用另外文章再做详细介绍。

HLS优势 

• 使用标准HTTP传输数据，具有较好的网络穿透及防屏蔽性，更易于内容分发网络传输。
• HLS协议本身是支持码率自适应的，客户端可以根据实际网络状况切换到合适的码率播放。
• HLS内容发布服务更简单，对系统设备要求较低，更容易实现负载均衡，并且HLS是无状态协议的HTTP，客户端只需要下载即可

HLS的格式

可简单归结如下：

网络协议	HTTP
封装格式	MEPG-2 TS
编码格式	视频编码格式为H.264/H.265，音频编码格式为MP3、AAC、AC-3或EC-3等
索引文件	M3U8

M3U8格式

相关名词

playlist：播放列表

Media Playlist：媒体播放列表（单一码率）

Variant Stream：同一份媒体资源的多份流列表资源（多码率其中一条流）

Master Playlist：主播放列表（多码率/有嵌套）

n_playlists：多个播放列表，即多个码率的m3u8

n_segments ：一个playlist对应的分片的个数（ts数）

n_variants：有几条不同的码流（至少一个）

segment：存放的是http+ts形式的url

variant：存放解析的metadata等数据

cur_seq_no：对应的是那个ts索引

is_segment： 是否是分片

is_variant： 是否有多码率

m3u8 文件实质是一个播放列表（playlist），其可能是一个媒体播放列表（Media Playlist），或者是一个主列表（Master Playlist）。但无论是哪种播放列表，其内部文字使用的都是 utf-8 编码。

        当 m3u8 文件作为媒体播放列表（Meida Playlist）时，其内部信息记录的是一系列媒体片段资源，顺序播放该片段资源，即可完整展示多媒体资源。其格式如下所示：

#EXTM3U
#EXT-X-TARGETDURATION:10

#EXTINF:9.009,
http://media.example.com/first.ts
#EXTINF:9.009,
http://media.example.com/second.ts
#EXTINF:3.003,
http://media.example.com/third.ts

  当 m3u8 作为主播放列表（Master Playlist）时，其内部提供的是同一份媒体资源的多份流列表资源（Variant Stream）。其格式如下所示：

#EXTM3U
#EXT-X-STREAM-INF:BANDWIDTH=150000,RESOLUTION=416x234,CODECS="avc1.42e00a,mp4a.40.2"
http://example.com/low/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=240000,RESOLUTION=416x234,CODECS="avc1.42e00a,mp4a.40.2"
http://example.com/lo_mid/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=440000,RESOLUTION=416x234,CODECS="avc1.42e00a,mp4a.40.2"
http://example.com/hi_mid/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=640000,RESOLUTION=640x360,CODECS="avc1.42e00a,mp4a.40.2"
http://example.com/high/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=64000,CODECS="mp4a.40.5"
http://example.com/audio/index.m3u8

带有广告的格式如下： 

#EXTM3U//m3u8标识，第一行必须是这个
#EXT-X-TARGETDURATION:10//切片最大duration ，下面4段ts 最大duration 是10， 因此这里为10
#EXT-X-VERSION:3//HLS version :V3
#EXT-X-MEDIA-SEQUENCE:0//第一个切片序列号为0
#EXTINF:10.0,
ad0.ts//广告0
#EXTINF:8.0,
ad1.ts//广告1
#EXT-X-DISCONTINUITY//当以下编码参数更改时file format/number and type of tracks/encoding 
//parameters/encoding sequence/timestamp sequence，需要在m3u8中加入#EXT-X-DISCONTINUITY隔离开，让播放器重新初始化。多用于插入广告
/*#EXT-X-DISCONTINUITY
当遇到该tag的时候说明以下属性发生了变化:
file format 
number and type of tracks
encoding parameters
encoding sequence
timestamp sequence
*/
#EXTINF:10.0,//分片时间
movieA.ts
#EXTINF:10.0,
movieB.ts

框架

通过两种方式表示

流程图

框架图

 代码分析

源码位于libavformat/hls.c 中

主要函数接口

AVInputFormat ff_hls_demuxer = {
    .name           = "hls,applehttp",
    .long_name      = NULL_IF_CONFIG_SMALL("Apple HTTP Live Streaming"),
    .priv_class     = &hls_class,
    .priv_data_size = sizeof(HLSContext),
    .flags          = AVFMT_NOGENSEARCH,
    .read_probe     = hls_probe, // 媒体格式探测
    .read_header    = hls_read_header, // 读取协议头并获取节目信息
    .read_packet    = hls_read_packet, // 读取音视频包
    .read_close     = hls_close, // 关闭HLS通信
    .read_seek      = hls_read_seek, // 实现HLS的seek操作
};

        FFmpeg中的demuxer，我们主要关注其read_probe、read_header、read_packet、read_close、read_seek这五个函数指针所对应的实现代码。FFmpeg在实现demuxer时将其协议解析的部分完整封装了demuxer中。

        其中HLSContext定义如下

typedef struct HLSContext {
    AVClass *class;
    AVFormatContext *ctx;
    int n_variants;
    struct variant **variants;/*以"EXT-X-STREAM-INF"为开头，多码率流 */
    int n_playlists;/* 存放个数 */
    struct playlist **playlists;/* 存放segment(ts url)的列表 */
    int n_renditions;
    struct rendition **renditions;
 
    int cur_seq_no;
    int live_start_index;/* 直播流开始段的索引（负值来自末尾） */
    int first_packet;//首个Packet
    int64_t first_timestamp;//首个packet的时间戳
    int64_t cur_timestamp;//当前时间戳
    AVIOInterruptCB *interrupt_callback;//avio的中断回调
    AVDictionary *avio_opts;//avio的options
    int strict_std_compliance;
    char *allowed_extensions;
    int max_reload;
    int http_persistent;
    int http_multiple;
    AVIOContext *playlist_pb;
} HLSContext;

hls_probe

probe探测流的format

static int hls_probe(AVProbeData *p)
{
    /* HLS协议要求必须以#EXTM3U打头，并至少有下面三个字段之一存在 */
    if (strncmp(p->buf, "#EXTM3U", 7))//第一行必须是#EXTM3U
        return 0;
 
    if (strstr(p->buf, "#EXT-X-STREAM-INF:")     ||
        strstr(p->buf, "#EXT-X-TARGETDURATION:") ||
        strstr(p->buf, "#EXT-X-MEDIA-SEQUENCE:"))//通过其他字段，进一步判断，有其一即可
        return AVPROBE_SCORE_MAX;
    return 0;
}

         通过判断第一行是否是“#EXTM3U”，如果是，继续判断是否有"#EXT-X-STREAM-INF:"，"#EXT-X-TARGETDURATION:"和"#EXT-X-MEDIA-SEQUENCE:"，三者有其一即可。

hls_read_header

        n_playlist 就是m3u8 文件的数组形式，segment 存放的是ts 的url。hls_read_header主要工作是解析m3u8，把segment存入n_playlists，然后初始化HLS，接着将得到的ts url 传入avformat_open_input() 中。

static int hls_read_header(AVFormatContext *s)
{
    void *u = (s->flags & AVFMT_FLAG_CUSTOM_IO) ? NULL : s->pb;
    HLSContext *c = s->priv_data;
    
    /*重点1： 解析m3u8 */
    if ((ret = parse_playlist(c, s->filename, NULL, s->pb)) < 0)
        goto fail;

    /* 保存avio的options */
    if ((ret = save_avio_options(s)) < 0)
        goto fail;

    /* 设置是否可以seek，即是否是直播，0为直播，1为点播 */
    av_dict_set(&c->avio_opts, "seekable", "0", 0);
    
    /* n_variants 必须要大于等于1 */
    if (c->n_variants == 0) {
        av_log(NULL, AV_LOG_WARNING, "Empty playlist\n");
        ret = AVERROR_EOF;
        goto fail;
    }
    /* 对于n_playlists，如果大于1，证明有多个码率的playlist，需要逐个解析其中的playlist */
    if (c->n_playlists > 1 || c->playlists[0]->n_segments == 0) {
        for (i = 0; i < c->n_playlists; i++) {
            struct playlist *pls = c->playlists[i];
            if ((ret = parse_playlist(c, pls->url, pls, NULL)) < 0)
                goto fail;
        }
    }
    /* n_segments 分片的个数*/
    if (c->variants[0]->playlists[0]->n_segments == 0) {
        av_log(NULL, AV_LOG_WARNING, "Empty playlist\n");
        ret = AVERROR_EOF;
        goto fail;
    }

    /* finished：1为点播，0为直播，如果不是直播流，则要计算流总的duration*/
    if (c->variants[0]->playlists[0]->finished) {
        int64_t duration = 0;
        for (i = 0; i < c->variants[0]->playlists[0]->n_segments; i++)
            duration += c->variants[0]->playlists[0]->segments[i]->duration;
        s->duration = duration;
    }

    /* renditions以"#EXT-X-MEDIA:"为标志，被用来在PlayList中表示相同内容的不用语种/译文的版本，比如可以通过使用3个这种tag表示3中不用语音的音频 */
    for (i = 0; i < c->n_variants; i++) {
        struct variant *var = c->variants[i];

        if (var->audio_group[0])
            add_renditions_to_variant(c, var, AVMEDIA_TYPE_AUDIO, var->audio_group);
        if (var->video_group[0])
            add_renditions_to_variant(c, var, AVMEDIA_TYPE_VIDEO, var->video_group);
        if (var->subtitles_group[0])
            add_renditions_to_variant(c, var, AVMEDIA_TYPE_SUBTITLE, var->subtitles_group);
    }

    /* 重点2：n_variants有几条不同的码流（至少一个），并为之创建programe*/
    for (i = 0; i < c->n_variants; i++) {
        struct variant *v = c->variants[i];
        AVProgram *program;

        program = av_new_program(s, i);
        if (!program)
            goto fail;
        //设置元辅助比特率属性值
        av_dict_set_int(&program->metadata, "variant_bitrate", v->bandwidth, 0);
    }

    /* 选择起始segment的索引cur_seq_no，并记录所有分片里最大的序号 highest_cur_seq_no */
    for (i = 0; i < c->n_playlists; i++) {
        struct playlist *pls = c->playlists[i];

        if (pls->n_segments == 0)
            continue;

        pls->cur_seq_no = select_cur_seq_no(c, pls);
        highest_cur_seq_no = FFMAX(highest_cur_seq_no, pls->cur_seq_no);
    }

    /* 重点3：对每个playlist打开其demuxer ，大于1则表明有多个码率的流，一个segment对应一个ts的url*/
    for (i = 0; i < c->n_playlists; i++) {
        struct playlist *pls = c->playlists[i];
        AVInputFormat *in_fmt = NULL;
        //创建context
        if (!(pls->ctx = avformat_alloc_context())) {
            ret = AVERROR(ENOMEM);
            goto fail;
        }
        //n_segments =0证明没有有ts的url
        if (pls->n_segments == 0)
            continue;

        pls->index  = i;
        pls->needed = 1;
        pls->parent = s;

        /* 调整直播流的起播索引号，以保证所有playlist是同步的,start_seq_no起始segment索引号,默认为0，cur_seq_no当前索引号*/
        if (!pls->finished && pls->cur_seq_no == highest_cur_seq_no - 1 &&
            highest_cur_seq_no < pls->start_seq_no + pls->n_segments) {
            pls->cur_seq_no = highest_cur_seq_no;
        }

        pls->read_buffer = av_malloc(INITIAL_BUFFER_SIZE);
        if (!pls->read_buffer){
            ret = AVERROR(ENOMEM);
            avformat_free_context(pls->ctx);
            pls->ctx = NULL;
            goto fail;
        }
        /* 这里初始化了AVIOContext，留意read_data函数，这将是后续读包的核心 */
        ffio_init_context(&pls->pb, pls->read_buffer, INITIAL_BUFFER_SIZE, 0, pls,
                          read_data, NULL, NULL);
        pls->pb.seekable = 0;
        //探测流的格式，用read_data读第0片，探测视频文件的解复用是AVInputFormat ff_mpegts_demuxer
        ret = av_probe_input_buffer(&pls->pb, &in_fmt, pls->segments[0]->url,
                                    NULL, 0, 0);
        if (ret < 0) {
            /* Free the ctx - it isn't initialized properly at this point,
             * so avformat_close_input shouldn't be called. If
             * avformat_open_input fails below, it frees and zeros the
             * context, so it doesn't need any special treatment like this. */
            av_log(s, AV_LOG_ERROR, "Error when loading first segment '%s'\n", pls->segments[0]->url);
            avformat_free_context(pls->ctx);
            pls->ctx = NULL;
            goto fail;
        }
        pls->ctx->pb       = &pls->pb;
        pls->ctx->io_open  = nested_io_open;

        if ((ret = ff_copy_whiteblacklists(pls->ctx, s)) < 0)
            goto fail;
        /* 接下来就是打开流，获取其中AVStream信息了，打开第0片视频文件 */
        ret = avformat_open_input(&pls->ctx, pls->segments[0]->url, in_fmt, NULL);
        if (ret < 0)
            goto fail;

        pls->has_noheader_flag = !!(pls->ctx->ctx_flags & AVFMTCTX_NOHEADER);

        /* 为这个播放列表中的每个流创建新的 AVStreams */
        ret = update_streams_from_subdemuxer(s, pls);
        if (ret < 0)
            goto fail;

        add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_AUDIO);
        add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_VIDEO);
        add_metadata_from_renditions(s, pls, AVMEDIA_TYPE_SUBTITLE);
    }

    update_noheader_flag(s);

    return 0;
fail:
    hls_close(s);
    return ret;
}

m3u8，playlist和segment的关系图

 下面我们分析下 parse_playlist()。此函数负责解析m3u8 文件， 并将信息记录到playlist 中。

在开始介绍前，我们先看一下playlist结构体的定义：

/* 每个playlist都有自己的demuxer，如果该playlist是在用的，它还会有AVIOContext和AVPacket */
struct playlist {
    char url[MAX_URL_SIZE];
    AVIOContext pb;
    uint8_t* read_buffer;
    AVIOContext *input;
    int input_read_done;
    AVIOContext *input_next;
    int input_next_requested;
    AVFormatContext *parent;// 这个将指向公用的AVFormatContext
    int index;
    AVFormatContext *ctx; // 这个将用于解析当前playlist的所有segment
    AVPacket pkt;
    int has_noheader_flag;
 
    /* 当前playlist中包含的AVStream信息 */
    AVStream **main_streams;
    int n_main_streams;
 
    int finished; /* segment读取状态的相关参数 */
    enum PlaylistType type;
    int64_t target_duration;
    int start_seq_no;//segment起始索引号，默认为0
    int n_segments; /* 当前playlist中的所有segment数组 */
    struct segment **segments;
    int needed;
    int cur_seq_no;//segment当前索引号,直播会实时更新
    int64_t cur_seg_offset;
    int64_t last_load_time;
 
    /* Media Initialization Section */
    struct segment *cur_init_section;
    uint8_t *init_sec_buf;
    unsigned int init_sec_buf_size;
    unsigned int init_sec_data_len;
    unsigned int init_sec_buf_read_offset;
 
    char key_url[MAX_URL_SIZE]; /* HLS解密密钥对应的URL */
    uint8_t key[16];
 
    int64_t seek_timestamp; /* seek相关的参数 */
    int seek_flags;
    int seek_stream_index; /* into subdemuxer stream array */
 
    /* 和当前playlist相关的Renditions（可选） */
    int n_renditions;
    struct rendition **renditions;
 
    /* Media Initialization Sections (EXT-X-MAP)（可选） */
    int n_init_sections;
    struct segment **init_sections;
};

parse_playlist 为下载m3u8，并解析m3u8

//is_segment 是否是分片，is_variant 是否有多码率
static int parse_playlist(HLSContext *c, const char *url,
                          struct playlist *pls, AVIOContext *in)
{
    int ret = 0, is_segment = 0, is_variant = 0;
    int64_t duration = 0;
    enum KeyType key_type = KEY_NONE;
    uint8_t iv[16] = "";
    int has_iv = 0;
    char key[MAX_URL_SIZE] = "";
    char line[MAX_URL_SIZE];
    const char *ptr;
    int close_in = 0;
    int64_t seg_offset = 0;
    int64_t seg_size = -1;
    uint8_t *new_url = NULL;
    struct variant_info variant_info;
    char tmp_str[MAX_URL_SIZE];
    struct segment *cur_init_section = NULL;
     /* 创建用于HTTP请求的AVIO，下载m3u8 
    m3u8内容保存在in的如下缓存中：
    unsigned char *buffer;  /**< Start of the buffer. */
    int buffer_size;        /**< Maximum buffer size */
    unsigned char *buf_ptr; /**< Current position in the buffer */
    unsigned char *buf_end; /**< End of the data, may be less than
    */
    if (!in) {
#if 1
        AVDictionary *opts = NULL;
        close_in = 1;
        /* Some HLS servers don't like being sent the range header */
        av_dict_set(&opts, "seekable", "0", 0);

        // broker prior HTTP options that should be consistent across requests
        av_dict_set(&opts, "user-agent", c->user_agent, 0);
        av_dict_set(&opts, "cookies", c->cookies, 0);
        av_dict_set(&opts, "headers", c->headers, 0);
        av_dict_set(&opts, "http_proxy", c->http_proxy, 0);
        //请求http,下载m3u8
        ret = c->ctx->io_open(c->ctx, &in, url, AVIO_FLAG_READ, &opts);
    }
    /* HTTP-URL重定向 */
    if (av_opt_get(in, "location", AV_OPT_SEARCH_CHILDREN, &new_url) >= 0)
        url = new_url;
    //获取一行
    read_chomp_line(in, line, sizeof(line));
    if (strcmp(line, "#EXTM3U")) {/* HLS协议标志起始头 */
        ret = AVERROR_INVALIDDATA;
        goto fail;
    }
    /* 释放已经存在的pls及segment */
    if (pls) {
        free_segment_list(pls);
        pls->finished = 0;
        pls->type = PLS_TYPE_UNSPECIFIED;
    }
    /* 以下是具体协议的解析 */
    while (!avio_feof(in)) {
        read_chomp_line(in, line, sizeof(line));
        if (av_strstart(line, "#EXT-X-STREAM-INF:", &ptr)) {//多码率标志
            is_variant = 1;
            memset(&variant_info, 0, sizeof(variant_info));
            ff_parse_key_value(ptr, (ff_parse_key_val_cb) handle_variant_args,
                               &variant_info);
        } else if (av_strstart(line, "#EXT-X-KEY:", &ptr)) {//是否加密
            struct key_info info = {{0}};
            ff_parse_key_value(ptr, (ff_parse_key_val_cb) handle_key_args,
                               &info);
            key_type = KEY_NONE;
            has_iv = 0;
            if (!strcmp(info.method, "AES-128"))
                key_type = KEY_AES_128;
            if (!strcmp(info.method, "SAMPLE-AES"))
                key_type = KEY_SAMPLE_AES;
            if (!strncmp(info.iv, "0x", 2) || !strncmp(info.iv, "0X", 2)) {
                ff_hex_to_data(iv, info.iv + 2);
                has_iv = 1;
            }
            av_strlcpy(key, info.uri, sizeof(key));
        } else if (av_strstart(line, "#EXT-X-MEDIA:", &ptr)) {//被用来在PlayList中表示相同内容的不用语种/译文的版本
            struct rendition_info info = {{0}};
            ff_parse_key_value(ptr, (ff_parse_key_val_cb) handle_rendition_args,
                               &info);
            new_rendition(c, &info, url);
        } else if (av_strstart(line, "#EXT-X-TARGETDURATION:", &ptr)) {//分片最大时间
            ret = ensure_playlist(c, &pls, url);
            if (ret < 0)
                goto fail;
            pls->target_duration = strtoll(ptr, NULL, 10) * AV_TIME_BASE;
        } else if (av_strstart(line, "#EXT-X-MEDIA-SEQUENCE:", &ptr)) {//每一个media URI 在 PlayList中只有唯一的序号，相邻之间序号+1，如果没有，默认为0
            ret = ensure_playlist(c, &pls, url);
            if (ret < 0)
                goto fail;
            pls->start_seq_no = atoi(ptr);/* 起始segment number */
        } else if (av_strstart(line, "#EXT-X-PLAYLIST-TYPE:", &ptr)) {
            ret = ensure_playlist(c, &pls, url);
            if (ret < 0)
                goto fail;
            if (!strcmp(ptr, "EVENT"))
                pls->type = PLS_TYPE_EVENT;
            else if (!strcmp(ptr, "VOD"))
                pls->type = PLS_TYPE_VOD;
        } else if (av_strstart(line, "#EXT-X-MAP:", &ptr)) {
            struct init_section_info info = {{0}};
            ret = ensure_playlist(c, &pls, url);
            if (ret < 0)
                goto fail;
            ff_parse_key_value(ptr, (ff_parse_key_val_cb) handle_init_section_args,
                               &info);
            cur_init_section = new_init_section(pls, &info, url);
        } else if (av_strstart(line, "#EXT-X-ENDLIST", &ptr)) {//是否直播
            if (pls)
                pls->finished = 1;//VOD
        } else if (av_strstart(line, "#EXTINF:", &ptr)) {
            is_segment = 1;
            duration   = atof(ptr) * AV_TIME_BASE;
        } else if (av_strstart(line, "#EXT-X-BYTERANGE:", &ptr)) {
            seg_size = strtoll(ptr, NULL, 10);
            ptr = strchr(ptr, '@');
            if (ptr)/* 使用字节划分的m3u8 */
                seg_offset = strtoll(ptr+1, NULL, 10);
        } else if (av_strstart(line, "#", NULL)) {
            continue; /* 忽略无法识别的字段 */
        } else if (line[0]) {
            if (is_variant) {/* 针对variant的处理，下一行一般是URL */
                if (!new_variant(c, &variant_info, line, url)) {
                    ret = AVERROR(ENOMEM);
                    goto fail;
                }
                is_variant = 0;
            }
            if (is_segment) {/*针对segment的处理，需要拼接segment的URL，下面是 segment数据*/
                struct segment *seg;
                if (!pls) {
                    if (!new_variant(c, 0, url, NULL)) {
                        ret = AVERROR(ENOMEM);
                        goto fail;
                    }
                    pls = c->playlists[c->n_playlists - 1];
                }
                //准备创建segment对象，保存TS片。
                seg = av_malloc(sizeof(struct segment));
                if (!seg) {
                    ret = AVERROR(ENOMEM);
                    goto fail;
                }
                seg->duration = duration;
                seg->key_type = key_type;
                if (has_iv) {
                    memcpy(seg->iv, iv, sizeof(iv));
                } else {
                    int seq = pls->start_seq_no + pls->n_segments;
                    memset(seg->iv, 0, sizeof(seg->iv));
                    AV_WB32(seg->iv + 12, seq);
                }

                if (key_type != KEY_NONE) {
                    ff_make_absolute_url(tmp_str, sizeof(tmp_str), url, key);
                    seg->key = av_strdup(tmp_str);
                    if (!seg->key) {
                        av_free(seg);
                        ret = AVERROR(ENOMEM);
                        goto fail;
                    }
                } else {
                    seg->key = NULL;
                }

                ff_make_absolute_url(tmp_str, sizeof(tmp_str), url, line);
                seg->url = av_strdup(tmp_str);//将url 传到此segment 节点中
                if (!seg->url) {
                    av_free(seg->key);
                    av_free(seg);
                    ret = AVERROR(ENOMEM);
                    goto fail;
                }
                //添加到segment array 中，即把TS片名称保存到链表
                dynarray_add(&pls->segments, &pls->n_segments, seg);
                is_segment = 0;

                seg->size = seg_size;
                if (seg_size >= 0) {
                    seg->url_offset = seg_offset;
                    seg_offset += seg_size;
                    seg_size = -1;
                } else {
                    seg->url_offset = 0;
                    seg_offset = 0;
                }

                seg->init_section = cur_init_section;
            }
        }
    }
    if (pls)
        pls->last_load_time = av_gettime_relative();

fail:
    av_free(new_url);
    if (close_in)
        ff_format_io_close(c->ctx, &in);
    return ret;
}

我们看一下如何选择起始segment的索引，select_cur_seq_no代码如下：

static int select_cur_seq_no(HLSContext *c, struct playlist *pls)
{
    int seq_no;
	/* 直播情况下，定期更新m3u8 */
    if (!pls->finished && !c->first_packet &&
        av_gettime_relative() - pls->last_load_time >= default_reload_interval(pls))
        parse_playlist(c, pls->url, pls, NULL);
 
    /* 对于非直播的情况，直接通过时长查找对应的segment索引号（seek时比较常用的逻辑） */
    if (pls->finished && c->cur_timestamp != AV_NOPTS_VALUE) {//cur_timestamp != AV_NOPTS_VALUE表明有seek
        find_timestamp_in_playlist(c, pls, c->cur_timestamp, &seq_no);
        return seq_no;
    }
 
    if (!pls->finished) {
        if (!c->first_packet && /* 是在播放中选择segment */
            c->cur_seq_no >= pls->start_seq_no &&
            c->cur_seq_no < pls->start_seq_no + pls->n_segments)
            return c->cur_seq_no;
 
        /* 直播情况下，需要参考live_start_index调整下 */
        if (c->live_start_index < 0)
            return pls->start_seq_no + FFMAX(pls->n_segments + c->live_start_index, 0);
        else
            return pls->start_seq_no + FFMIN(c->live_start_index, pls->n_segments - 1);
    }
 
    /* 其他情况直接返回起始segment索引号*/
    return pls->start_seq_no;
}

主要三个步骤

1）创建http请求，下载m3u8，并解析m3u8，把多码率的流保存到n_playlist，并对每一个playlist进行解析，把ts的url保存入segments

2）

hls_read_packet

该函数主要功能是读取原始的数据，并解析为AVPacket。

流程图

 代码分析

/*
seek也在这个函数里有操作，当seek之后第一帧的时间戳一定是等于或者晚于我们要期望的seek时间的
*/
static int hls_read_packet(AVFormatContext *s, AVPacket *pkt)
{
    HLSContext *c = s->priv_data;
    int ret, i, minplaylist = -1;
	/* 这个函数会根据AVStream.discard标志来判断关闭不需要的HTTP下载 */
    recheck_discard_flags(s, c->first_packet);
    c->first_packet = 0;
 
    for (i = 0; i < c->n_playlists; i++) {
        struct playlist *pls = c->playlists[i];
        /* 从每一个已打开的playlist上读取一个AVPacket */
        if (pls->needed && !pls->pkt.data) {
            while (1) {
                int64_t ts_diff;
                AVRational tb;
                ret = av_read_frame(pls->ctx, &pls->pkt);
                if (ret < 0) {
                    if (!avio_feof(&pls->pb) && ret != AVERROR_EOF)
                        return ret;
                    reset_packet(&pls->pkt);
                    break;
                } else {
                    if (c->first_timestamp == AV_NOPTS_VALUE &&
                        pls->pkt.dts       != AV_NOPTS_VALUE)
                        c->first_timestamp = av_rescale_q(pls->pkt.dts,
                            get_timebase(pls), AV_TIME_BASE_Q);
                }
                //没有SEEK的话，读包成功直接退出循环				
                if (pls->seek_timestamp == AV_NOPTS_VALUE)
                    break;
                 /* 以下部分跟seek有关，无seek操作无需后续判断 */
                if (pls->seek_stream_index < 0 ||
                    pls->seek_stream_index == pls->pkt.stream_index) {
 
                    if (pls->pkt.dts == AV_NOPTS_VALUE) {
                        pls->seek_timestamp = AV_NOPTS_VALUE;
                        break;
                    }
 
                    tb = get_timebase(pls);
                    // 得到当前帧的时间戳和要seek的时间戳的差值
                    ts_diff = av_rescale_rnd(pls->pkt.dts, AV_TIME_BASE,
                                            tb.den, AV_ROUND_DOWN) -
                            pls->seek_timestamp;
                    //本片中通过比较pkt.dts，seek_timestamp接近了SEEK点，本次SEEK成功
                    // seek_flags就支持AVSEEK_FLAG_ANY和AV_PKT_FLAG_KEY，返回的帧一定是要晚于seek时间戳的
                    if (ts_diff >= 0 && (pls->seek_flags  & AVSEEK_FLAG_ANY ||
                                        pls->pkt.flags & AV_PKT_FLAG_KEY)) {
                        pls->seek_timestamp = AV_NOPTS_VALUE;
                        break;
                    }
                }/* 到此，说面当前AVPacket是需要丢弃的，重新读取 */
                av_packet_unref(&pls->pkt);
                reset_packet(&pls->pkt);
            }
        }
        /* 从所有AVStream中找到dts最小的一个，记录该索引值 */
        if (pls->pkt.data) {
            struct playlist *minpls = minplaylist < 0 ?
                                     NULL : c->playlists[minplaylist];
            if (minplaylist < 0) {
                minplaylist = i;
            } else {
                int64_t dts     =    pls->pkt.dts;
                int64_t mindts  = minpls->pkt.dts;
 
                if (dts == AV_NOPTS_VALUE ||
                    (mindts != AV_NOPTS_VALUE && compare_ts_with_wrapdetect(dts, pls, mindts, minpls) < 0))
                    minplaylist = i;
            }
        }
    }
 
    /* 成功读取AVPacket，需要返回给上层调用者 */
    if (minplaylist >= 0) {
        struct playlist *pls = c->playlists[minplaylist];
        AVStream *ist;
        AVStream *st;
 
		/* 判断stream_index有效性 */
        if (pls->pkt.stream_index >= pls->n_main_streams) {
            av_packet_unref(&pls->pkt);
            reset_packet(&pls->pkt);
            return AVERROR_BUG;
        }
 
        ist = pls->ctx->streams[pls->pkt.stream_index];
        st = pls->main_streams[pls->pkt.stream_index];
 
        *pkt = pls->pkt;
        pkt->stream_index = st->index;
        reset_packet(&c->playlists[minplaylist]->pkt);
		/* 更新当前HLS的读取位置 */
        if (pkt->dts != AV_NOPTS_VALUE)
            c->cur_timestamp = av_rescale_q(pkt->dts,
                                            ist->time_base,
                                            AV_TIME_BASE_Q);
 
        return 0;
    }
    return AVERROR_EOF;
}

 从这个代码上来看很奇怪，这里没有包含任何http请求、playlist更新、segment的处理逻辑，那么真正的逻辑隐藏在哪里呢？不知道大家还记得2.2节中hls_read_header函数中对AVIOContext的初始化逻辑，其实最终的解析主要部分就在其中一个不起眼的函数中——read_data。

static int read_data(void *opaque, uint8_t *buf, int buf_size)
{
    struct playlist *v = opaque;
    HLSContext *c = v->parent->priv_data;
    int ret;
    int just_opened = 0;
    int reload_count = 0;
    struct segment *seg;
 
restart: /* 标签：重启新的下载 */
    if (!v->needed)
        return AVERROR_EOF;
     //v->input 判断本片是否打开过
    if (!v->input || (c->http_persistent && v->input_read_done)) {
        int64_t reload_interval;
 
        /* 检查playlist是否需要下载，如果不需要了，直接返回EOF */
        if (v->ctx && v->ctx->nb_streams) {
            v->needed = 0;
            for (i = 0; i < v->n_main_streams; i++) {
                if (v->main_streams[i]->discard < AVDISCARD_ALL) {
                    v->needed = 1;
                    break;
                }
            }
        }
        //校验playlist合法性
        v->needed = playlist_needed(v);//新版本

        if (!v->needed) {
            return AVERROR_EOF;
        }
 
        /* 对于直播流，定期更新playlist */
        reload_interval = default_reload_interval(v);
 
reload: /* 标签：重新下载playlist */
        reload_count++;
        if (reload_count > c->max_reload)
            return AVERROR_EOF;
//如果是直播流的情况下，并且已经到达了reload的时机，重新去刷新playlist获取ts列表
        if (!v->finished &&
            av_gettime_relative() - v->last_load_time >= reload_interval) {
            //parse_playlist为下载，解析m3u8。
            if ((ret = parse_playlist(c, v->url, v, NULL)) < 0)
                return ret;
 
            /* 按照HLS协议规定，如果还需要请求playlist，请求间隔可以设置为segment时长的一半 */
            reload_interval = v->target_duration / 2;
        }
//如果当前的seqnumer，比起始的seq都要小，那么直接跳过中间segment并刷新到最新数据。
        if (v->cur_seq_no < v->start_seq_no) {
            //一开始，cur_seq_no为pls->start_seq_no +FFMAX(pls->n_segments -3, 0)。
            /* 这种情况下客户端因为某些原因下载过慢，直接调整cur_seq_no，跳过一些segment */
            v->cur_seq_no = v->start_seq_no;
        }
//如果当前的的序号>起始序号+总数ts片数量。
        if (v->cur_seq_no >= v->start_seq_no + v->n_segments) {
            //每下一个TS片，cur_seq_no加1，当太大的时，跳到reload，即：重新获取m3u8。
            if (v->finished) /* 点播的情况下，表示读取完成 */
                return AVERROR_EOF;
//如果是直播，如果还没有到达reload时间，那么就sleep一直等到reload时间
            while (av_gettime_relative() - v->last_load_time < reload_interval) {
                if (ff_check_interrupt(c->interrupt_callback))
                    return AVERROR_EXIT;
                av_usleep(100*1000);
            }
            /* 上面的循环用于等待给定的时间，然后重新加载playlist */
            goto reload;
        }
		/* 所有的前提都满足，可以开始读segment了 */
        v->input_read_done = 0;
        seg = current_segment(v);
 
        /* 如果存在，加载并更新Media Initialization Section */
        ret = update_init_section(v, seg);
        if (ret)
            return ret;
		
        if (c->http_multiple == 1 && v->input_next_requested) {
            FFSWAP(AVIOContext *, v->input, v->input_next);
            v->input_next_requested = 0;
            ret = 0;
        } else { /* 这里发起了新的HTTP请求 ,通过http下载ts的内容*/
            ret = open_input(c, v, seg, &v->input);
        }
        if (ret < 0) { /* 请求失败的情况下的处理，默认直接跳过当前segment */
            if (ff_check_interrupt(c->interrupt_callback))
                return AVERROR_EXIT;
            v->cur_seq_no += 1;
            goto reload;
        }
        just_opened = 1;
    }
 
    if (c->http_multiple == -1) {
        uint8_t *http_version_opt = NULL;
        int r = av_opt_get(v->input, "http_version", AV_OPT_SEARCH_CHILDREN, &http_version_opt);
        if (r >= 0) {
            c->http_multiple = strncmp((const char *)http_version_opt, "1.1", 3) == 0;
            av_freep(&http_version_opt);
        }
    }
 
    seg = next_segment(v); /* 这是使用单HTTP发起请求的处理 */
    if (c->http_multiple == 1 && !v->input_next_requested &&
        seg && seg->key_type == KEY_NONE && av_strstart(seg->url, "http", NULL)) {
        ret = open_input(c, v, seg, &v->input_next);
        if (ret < 0) {
            if (ff_check_interrupt(c->interrupt_callback))
                return AVERROR_EXIT;
            av_log(v->parent, AV_LOG_WARNING, "Failed to open segment %d of playlist %d\n",
                   v->cur_seq_no + 1,
                   v->index);
        } else {
            v->input_next_requested = 1;
        }
    }
 
    if (v->init_sec_buf_read_offset < v->init_sec_data_len) {
        /* 在任何实际数据返回之前，首先返回init section（解码器可能会依赖这些数据做初始化） */
        int copy_size = FFMIN(v->init_sec_data_len - v->init_sec_buf_read_offset, buf_size);
        memcpy(buf, v->init_sec_buf, copy_size);
        v->init_sec_buf_read_offset += copy_size;
        return copy_size;
    }
 
    seg = current_segment(v);
	/* 这是实际通过HTTP读取buf_size长度的数据，这个数据是交织后的数据，从server上获取的
    读出ts的部分内容到buf。*/
    ret = read_from_url(v, seg, buf, buf_size);
    if (ret > 0)//读出部分内容，就返回
        return ret;
	
	/* 读取完成了，可以关闭segment所使用的HTTP资源了 */
    if (c->http_persistent &&
        seg->key_type == KEY_NONE && av_strstart(seg->url, "http", NULL)) {
        v->input_read_done = 1;
    } else {//本片下载完
        ff_format_io_close(v->parent, &v->input);
    }
 
	/* 更新当前读取的seg_no，同时更新全局的，保证流切换之后可以同步 */
    v->cur_seq_no++;
 
    c->cur_seq_no = v->cur_seq_no;
 
    goto restart; /* 回到上面继续循环了 */
}

open_input 

static int open_input(HLSContext *c, struct playlist *pls, struct segment *seg)
{
    AVDictionary *opts = NULL;
    int ret;
    int is_http = 0;

    // broker prior HTTP options that should be consistent across requests
    av_dict_set(&opts, "user-agent", c->user_agent, 0);
    av_dict_set(&opts, "cookies", c->cookies, 0);
    av_dict_set(&opts, "headers", c->headers, 0);
    av_dict_set(&opts, "http_proxy", c->http_proxy, 0);
    av_dict_set(&opts, "seekable", "0", 0);

    if (seg->size >= 0) {
        /* try to restrict the HTTP request to the part we want
         * (if this is in fact a HTTP request) */
        av_dict_set_int(&opts, "offset", seg->url_offset, 0);
        av_dict_set_int(&opts, "end_offset", seg->url_offset + seg->size, 0);
    }
    //终于看到熟悉的log了
    av_log(pls->parent, AV_LOG_VERBOSE, "HLS request for url '%s', offset %"PRId64", playlist %d\n",
           seg->url, seg->url_offset, pls->index);

    if (seg->key_type == KEY_NONE) {//http请求
        ret = open_url(pls->parent, &pls->input, seg->url, c->avio_opts, opts, &is_http);
    } else if (seg->key_type == KEY_AES_128) {
        AVDictionary *opts2 = NULL;
        char iv[33], key[33], url[MAX_URL_SIZE];
        if (strcmp(seg->key, pls->key_url)) {
            AVIOContext *pb;
            if (open_url(pls->parent, &pb, seg->key, c->avio_opts, opts, NULL) == 0) {
                ret = avio_read(pb, pls->key, sizeof(pls->key));
                if (ret != sizeof(pls->key)) {
                    av_log(NULL, AV_LOG_ERROR, "Unable to read key file %s\n",
                           seg->key);
                }
                ff_format_io_close(pls->parent, &pb);
            } else {
                av_log(NULL, AV_LOG_ERROR, "Unable to open key file %s\n",
                       seg->key);
            }
            av_strlcpy(pls->key_url, seg->key, sizeof(pls->key_url));
        }
        ff_data_to_hex(iv, seg->iv, sizeof(seg->iv), 0);
        ff_data_to_hex(key, pls->key, sizeof(pls->key), 0);
        iv[32] = key[32] = '\0';
        if (strstr(seg->url, "://"))
            snprintf(url, sizeof(url), "crypto+%s", seg->url);
        else
            snprintf(url, sizeof(url), "crypto:%s", seg->url);

        av_dict_copy(&opts2, c->avio_opts, 0);
        av_dict_set(&opts2, "key", key, 0);
        av_dict_set(&opts2, "iv", iv, 0);

        ret = open_url(pls->parent, &pls->input, url, opts2, opts, &is_http);

        av_dict_free(&opts2);

        if (ret < 0) {
            goto cleanup;
        }
        ret = 0;
    } else if (seg->key_type == KEY_SAMPLE_AES) {
        av_log(pls->parent, AV_LOG_ERROR,
               "SAMPLE-AES encryption is not supported yet\n");
        ret = AVERROR_PATCHWELCOME;
    }
    else
      ret = AVERROR(ENOSYS);

    /* Seek to the requested position. If this was a HTTP request, the offset
     * should already be where want it to, but this allows e.g. local testing
     * without a HTTP server.
     *
     * This is not done for HTTP at all as avio_seek() does internal bookkeeping
     * of file offset which is out-of-sync with the actual offset when "offset"
     * AVOption is used with http protocol, causing the seek to not be a no-op
     * as would be expected. Wrong offset received from the server will not be
     * noticed without the call, though.
     */
    if (ret == 0 && !is_http && seg->key_type == KEY_NONE && seg->url_offset) {
        int64_t seekret = avio_seek(pls->input, seg->url_offset, SEEK_SET);
        if (seekret < 0) {
            av_log(pls->parent, AV_LOG_ERROR, "Unable to seek to offset %"PRId64" of HLS segment '%s'\n", seg->url_offset, seg->url);
            ret = seekret;
            ff_format_io_close(pls->parent, &pls->input);
        }
    }

cleanup:
    av_dict_free(&opts);
    pls->cur_seg_offset = 0;
    return ret;
}

其中open_url在做分析，url主要形式就是http+ts。顾名思义，就是要打开url

static int open_url(AVFormatContext *s, AVIOContext **pb, const char *url,
                    AVDictionary *opts, AVDictionary *opts2, int *is_http)
{
    HLSContext *c = s->priv_data;
    AVDictionary *tmp = NULL;
    const char *proto_name = NULL;
    int ret;

    av_dict_copy(&tmp, opts, 0);
    av_dict_copy(&tmp, opts2, 0);

    if (av_strstart(url, "crypto", NULL)) {//是否加密
        if (url[6] == '+' || url[6] == ':')
            proto_name = avio_find_protocol_name(url + 7);
    }

    if (!proto_name)
        proto_name = avio_find_protocol_name(url);//find对应的protocol，即http

    if (!proto_name)
        return AVERROR_INVALIDDATA;

    //仅支持http和file格式的protocol
    if (!av_strstart(proto_name, "http", NULL) && !av_strstart(proto_name, "file", NULL))
        return AVERROR_INVALIDDATA;
    if (!strncmp(proto_name, url, strlen(proto_name)) && url[strlen(proto_name)] == ':')
        ;
    else if (av_strstart(url, "crypto", NULL) && !strncmp(proto_name, url + 7, strlen(proto_name)) && url[7 + strlen(proto_name)] == ':')
        ;
    else if (strcmp(proto_name, "file") || !strncmp(url, "file,", 5))
        return AVERROR_INVALIDDATA;
    //根据protocol，会调用http_open，即http请求
    ret = s->io_open(s, pb, url, AVIO_FLAG_READ, &tmp);
    if (ret >= 0) {
        // update cookies on http response with setcookies.
        void *u = (s->flags & AVFMT_FLAG_CUSTOM_IO) ? NULL : s->pb;
        update_options(&c->cookies, "cookies", u);
        av_dict_set(&opts, "cookies", c->cookies, 0);
    }

    av_dict_free(&tmp);
    //根据proto_name，判断是否是http
    if (is_http)
        *is_http = av_strstart(proto_name, "http", NULL);

    return ret;
}

        这是需要强调下，read_data就是完成一个I/O该做的事情，从server上去取数据，放到缓冲区中，真正的数据解析是由FFmpeg的通用框架实现的。换句话来说，HLS demuxer仅仅做了HLS协议相关的解析，关于mpeg-ts/mp4的解析实际上有它们各自的demuxer完成的。

hls_read_seek

• 它支持的flag只有AVSEEK_FLAG_BACKWARD和AVSEEK_FLAG_ANY
• 如果同时存在音频和视频，stream_index要使用视频的stream，否则AVSEEK_FLAG_BACKWARD这个flag相当于无效，因为音频的每一帧都是关键帧
• 它的工作就是根据当前的时间戳重新选择一个适合的ts文件（URL），主要关注这两个参数cur_seq_no和seek_timestamp
• 当下次调用hls_read_packet的时候就会调用http读取cur_seq_no上的流，一直读到符合当前seek_timestamp和seek_flags的流才返回，所以seek之后第一次读数据会比较慢

static int hls_read_seek(AVFormatContext *s, int stream_index,
                               int64_t timestamp, int flags)
{
    HLSContext *c = s->priv_data;
    struct playlist *seek_pls = NULL;
    int i, seq_no;
    int j;
    int stream_subdemuxer_index;
    int64_t first_timestamp, seek_timestamp, duration;
     // hls不支持byte方式的seek，并判断是否支持seek，hls的seek状态会动态改变的
    if ((flags & AVSEEK_FLAG_BYTE) || (c->ctx->ctx_flags & AVFMTCTX_UNSEEKABLE))
        return AVERROR(ENOSYS);
     
    first_timestamp = c->first_timestamp == AV_NOPTS_VALUE ?
                      0 : c->first_timestamp;
     // 计算以time_base为倍数的时间戳
//根据传入的timestamp，和index，转换最终的seek时间戳
    seek_timestamp = av_rescale_rnd(timestamp, AV_TIME_BASE,
                                    s->streams[stream_index]->time_base.den,
                                    flags & AVSEEK_FLAG_BACKWARD ?
                                    AV_ROUND_DOWN : AV_ROUND_UP);
 
    duration = s->duration == AV_NOPTS_VALUE ?
               0 : s->duration;
	/* 检查seek位置的有效性 */
    if (0 < duration && duration < seek_timestamp - first_timestamp)
        return AVERROR(EIO);
 
    /* 找到stream_index对应的playlist */
    for (i = 0; i < c->n_playlists; i++) {
        struct playlist *pls = c->playlists[i];
        for (j = 0; j < pls->n_main_streams; j++) {
            if (pls->main_streams[j] == s->streams[stream_index]) {
                seek_pls = pls;
                stream_subdemuxer_index = j;
                break;
            }
        }
    }
    /* 检查给定的seek timestamp对指定的流是否有效 */
//根据timestamp在playlist查找是第几片
    if (!seek_pls || !find_timestamp_in_playlist(c, seek_pls, seek_timestamp, &seq_no))
        return AVERROR(EIO);
 
    /* 所有参数都有效了，这就可以设置目标位置了 */
    // read_data会调用current_segment方法，下次下载第seq_no片
    seek_pls->cur_seq_no = seq_no;
    seek_pls->seek_stream_index = stream_subdemuxer_index;
	/* 下面是对正在读取的流的处理 */
    for (i = 0; i < c->n_playlists; i++) {
        struct playlist *pls = c->playlists[i];
        if (pls->input)//关闭现在正在下载的片input
            ff_format_io_close(pls->parent, &pls->input);
        pls->input_read_done = 0;
        if (pls->input_next)
            ff_format_io_close(pls->parent, &pls->input_next);
        pls->input_next_requested = 0;
        av_packet_unref(&pls->pkt);
        reset_packet(&pls->pkt);
        pls->pb.eof_reached = 0;
        /* 清空所有缓存的数据 */
        pls->pb.buf_end = pls->pb.buf_ptr = pls->pb.buffer;
        /* 重置读取位置，以确保demuxer知道需要seek */
        pls->pb.pos = 0;
        /* 清空subdemuxer缓存的所有AVPacket队列 */
        ff_read_frame_flush(pls->ctx);
         //hls_read_packet 里面会根据这个seek_timestamp判断下载seq_no片中，第多少帧刚好是这个SEEK点
        pls->seek_timestamp = seek_timestamp;
        pls->seek_flags = flags;
 /* 对于不是seek的playlist进行处理 */
        if (pls != seek_pls) {
            /* 对于不是seek的playlist，将其读取位置设置到seek目标点的位置 */
            find_timestamp_in_playlist(c, pls, seek_timestamp, &pls->cur_seq_no);
            pls->seek_stream_index = -1; /* 这两个标志将在read_packet中使用 */
            pls->seek_flags |= AVSEEK_FLAG_ANY;
        }
    }
	/* 最后记录seek目标位置 */
    c->cur_timestamp = seek_timestamp;
 
    return 0;
}

需要注意的是其中的第四个参数flag，表示seek到具体帧的方式。

#deflagfine AVSEEK_FLAG_BACKWARD    1 // 是seek到请求的timestamp之前最近的关键帧
#define AVSEEK_FLAG_BYTE            2 // 是基于字节位置的查找
#define AVSEEK_FLAG_ANY             4 //是可以seek到任意帧，注意不一定是关键帧，因此使用时可能会导致花屏 
#define AVSEEK_FLAG_FRAME           8 //是基于帧数量快进

关于hls seek不精准的问题

要做精准seek必定是需要修改hls_read_packet的代码的，要让它返回我们期望的时间戳的前一个关键帧的时间戳。

segment(http+ts)解析

结构上

AVFormatContext {
    AVIOContext *pb; {
        void *opaque -->    URLContext {  // 由ffurl_open()得到
                               struct URLProtocol *prot {  // http即为此类型
                                   const char *name;
                                   int (*url_open) (URLContext * h, const char *url, int flags);
                                   int (*url_read) (URLContext * h, char *buf, int size);
                               }
                            }
        int (*read_packet)(void *opaque, uint8_t *buf, int buf_size);
            --> ffurl_read(URLContext *h, unsigned char *buf, int size)
    }
    struct AVInputFormat *iformat; {  // ff_mpegts_demuxer即为此类型
        int (*read_packet)(struct AVFormatContext *, AVPacket *pkt);  --> mpegts_read_packet()
    }
}

调用关系上

av_read_packet(AVFormatContext *s, AVPacket *pkt)
|   // 通过: s->iformat->read_packet(s, pkt)调到:
+-> mpegts_read_packet(AVFormatContext *s, AVPacket *pkt)
    |
    +-> handle_packets(ts, 0)
        |   // 通过调用 static read_packet 调用:
        +-> avio_read(AVIOContext *pb, buf, TS_PACKET_SIZE);  // 该函数自带缓存buffer, 当buffer读完才去调用下面的s->read_packet()
            |
            +-> s->read_packet(s->opaque, buf, size)
avformat_open_input()
|
+-> init_input(AVFormatContext s, filename, &tmp)
    |
    +-> avio_open2(AVIOContext **s, *filename, int flags, *int_cb, **options)  
    |   |
    |   +-> ffurl_open(URLContext **puc, *filename, int flags, *int_cb, **options)
    |   |   |
    |   |   +-> ffurl_alloc(URLContext **puc, const char *filename, int flags, *int_cb) // 可以得到输入协议类型, 比如http.
    |   |   +-> ffurl_connect(URLContext* uc, AVDictionary **options)
    |   |       |
    |   |       +-> uc->prot->url_open(uc, uc->filename, uc->flags); // 对于http即为: http_open() :
    |   |           http_open(URLContext *h, const char *uri, int flags)
    |   |           |
    |   |           +-> http_open_cnx(URLContext *h)
    |   |               |
    |   |               +-> ffurl_open(URLContext **puc, *filename, int flags, *int_cb, **options) // 此时的filename已经变成tcp://172.16.5.144:5100
    |   +-> ffio_fdopen(AVIOContext **s, URLContext *h) // 在此分配: buffer(默认32k), AVIOContext, 并关联到URLContext
    |
    +-> av_probe_input_buffer(AVIOContext *pb, AVInputFormat **fmt, *filename, *logctx, offset, max_probe_size)
            // 如果是http播放在这里获取到mpegts等封装格式并存入s->iformat.
av_read_packet(AVFormatContext *s, AVPacket *pkt)
|
+-> s->iformat->read_packet(s, pkt);

问题：播放 http://127.0.0.1/streaming/output.m3u8  与播放 http://127.0.0.1/streaming/output1.ts 有何区别？

答： 通过上述两个url，我们可以得知前者是hls 协议，后者是http 协议。 对于后者通过ffmpeg 直接下载http 网络数据，然后去demux。 前者需在后者的基础上 通过m3u8文本解析出ts 流的url。 因此我们可以概述：相比与http 协议，hls 多了一个解析m3u8 文件的过程。  本文重点是在 hls.c 是如何解析m3u8 文件，并如何处理。让我们带着问题继续往下读。
 

问题：m3u8索引文件是什么地方开始请求拿到的？

答：

问题：codeblocks可以读取程序读写堆栈的过程，以下是HLS获取m3u8的过程

读m3u8的堆栈

请求ts的堆栈

读ts的堆栈

HTTP请求hls过程报文

HTTP报文件：
 
(3671754)request: GET /2017127/5de/529/zehBOsrZo57vbeJy/index.m3u8 HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Range: bytes=0-
Connection: close
Host: xxxcdn.hls.yyy.tv
Icy-MetaData: 1
[http @ 0x102708000] (3671754)header='HTTP/1.1 206 Partial Content'
[http @ 0x102708000] (3671754)header='Server: Tengine'
[http @ 0x102708000] (3671754)header='Content-Type: application/x-mpegURL'
[http @ 0x102708000] (3671754)header='Content-Length: 36072'
[http @ 0x102708000] (3671754)header='Connection: close'
[http @ 0x102708000] (3671754)header='Date: Thu, 07 Dec 2017 13:20:30 GMT'
[http @ 0x102708000] (3671754)header='x-oss-request-id: 5A29401E5FB1640973AC7221'
[http @ 0x102708000] (3671754)header='Accept-Ranges: bytes'
[http @ 0x102708000] (3671754)header='ETag: "5DBE181ACD4B983E026E19C50BFF23DD"'
[http @ 0x102708000] (3671754)header='Last-Modified: Thu, 07 Dec 2017 13:20:04 GMT'
[http @ 0x102708000] (3671754)header='x-oss-object-type: Normal'
[http @ 0x102708000] (3671754)header='x-oss-hash-crc64ecma: 17784356933318823809'
[http @ 0x102708000] (3671754)header='x-oss-storage-class: Standard'
[http @ 0x102708000] (3671754)header='Content-MD5: Xb4YGs1LmD4CbhnFC/8j3Q=='
[http @ 0x102708000] (3671754)header='x-oss-server-time: 3'
[http @ 0x102708000] (3671754)header='Via: cache25.l2nu20-2[0,200-0,H], cache1.l2nu20-2[0,0], cache8.cn307[0,206-0,H], cache6.cn307[3,0]'
[http @ 0x102708000] (3671754)header='Age: 763084'
[http @ 0x102708000] (3671754)header='X-Cache: HIT TCP_HIT dirn:17:96357170 mlen:-1'
[http @ 0x102708000] (3671754)header='X-Swift-SaveTime: Thu, 07 Dec 2017 15:58:04 GMT'
[http @ 0x102708000] (3671754)header='X-Swift-CacheTime: 2592000'
[http @ 0x102708000] (3671754)header='Content-Range: bytes 0-36071/36072'
[http @ 0x102708000] (3671754)header='access-control-allow-origin: *'
[http @ 0x102708000] (3671754)header='Timing-Allow-Origin: *'
[http @ 0x102708000] (3671754)header='EagleId: 7b81d7ce15134159145938051e'
[http @ 0x102708000] (3671754)header=''
[hls,applehttp @ 0x104007000] Format hls,applehttp probed with size=2048 and score=100
[hls,applehttp @ 0x104007000] Opening 'http://xxxcdn.hls.yyy.tv/2017127/5de/529/zehBOsrZo57vbeJy/0.ts' for reading
[http @ 0x1026275e0] 
(3671754)request: GET /2017127/5de/529/zehBOsrZo57vbeJy/0.ts HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: xxxcdn.hls.yyy.tv
Icy-MetaData: 1
 
 
[http @ 0x1026275e0] (3671754)header='HTTP/1.1 200 OK'
[http @ 0x1026275e0] (3671754)header='Server: Tengine'
[http @ 0x1026275e0] (3671754)header='Content-Type: video/MP2T'
[http @ 0x1026275e0] (3671754)header='Content-Length: 1233656'
[http @ 0x1026275e0] (3671754)header='Connection: close'
[http @ 0x1026275e0] (3671754)header='Date: Thu, 07 Dec 2017 13:20:31 GMT'
[http @ 0x1026275e0] (3671754)header='x-oss-request-id: 5A29401FF43DB73D87B40E71'
[http @ 0x1026275e0] (3671754)header='Accept-Ranges: bytes'
[http @ 0x1026275e0] (3671754)header='ETag: "8C8A9CAC5FC40404D629A2AF19B84A6F"'
[http @ 0x1026275e0] (3671754)header='Last-Modified: Thu, 07 Dec 2017 10:58:38 GMT'
[http @ 0x1026275e0] (3671754)header='x-oss-object-type: Normal'
[http @ 0x1026275e0] (3671754)header='x-oss-hash-crc64ecma: 13525335145074164730'
[http @ 0x1026275e0] (3671754)header='x-oss-storage-class: Standard'
[http @ 0x1026275e0] (3671754)header='Content-MD5: jIqcrF/EBATWKaKvGbhKbw=='
[http @ 0x1026275e0] (3671754)header='x-oss-server-time: 29'
[http @ 0x1026275e0] (3671754)header='Via: cache42.l2nu20-2[0,200-0,H], cache2.l2nu20-2[0,0], cache2.cn307[0,200-0,H], cache8.cn307[3,0]'
[http @ 0x1026275e0] (3671754)header='Age: 763086'
[http @ 0x1026275e0] (3671754)header='X-Cache: HIT TCP_HIT dirn:15:118266541 mlen:-1'
[http @ 0x1026275e0] (3671754)header='X-Swift-SaveTime: Thu, 07 Dec 2017 13:21:58 GMT'
[http @ 0x1026275e0] (3671754)header='X-Swift-CacheTime: 2592000'
[http @ 0x1026275e0] (3671754)header='access-control-allow-origin: *'
[http @ 0x1026275e0] (3671754)header='Timing-Allow-Origin: *'
[http @ 0x1026275e0] (3671754)header='EagleId: 7b81d7d015134159179953085e'
[http @ 0x1026275e0] (3671754)header=''
..........................................
.......................................... //下载0.ts文件
Format mpegts probed with size=2048 and score=50
(3671754)request: GET /2017127/5de/529/zehBOsrZo57vbeJy/1.ts HTTP/1.1
User-Agent: Lavf/57.41.100
Accept: */*
Connection: close
Host: xxxcdn.hls.yyy.tv
Icy-MetaData: 1
[http @ 0x102509500] (3671754)header='HTTP/1.1 200 OK'
[http @ 0x102509500] (3671754)header='Server: Tengine'
[http @ 0x102509500] (3671754)header='Content-Type: video/MP2T'
[http @ 0x102509500] (3671754)header='Content-Length: 1162028'
[http @ 0x102509500] (3671754)header='Connection: close'
[http @ 0x102509500] (3671754)header='Date: Thu, 07 Dec 2017 13:20:31 GMT'
[http @ 0x102509500] (3671754)header='x-oss-request-id: 5A29401F83B4CE0AA7B40025'
[http @ 0x102509500] (3671754)header='Accept-Ranges: bytes'
[http @ 0x102509500] (3671754)header='ETag: "95EFDEADB86482C551F1BCF868ECEA41"'
[http @ 0x102509500] (3671754)header='Last-Modified: Thu, 07 Dec 2017 10:58:49 GMT'
[http @ 0x102509500] (3671754)header='x-oss-object-type: Normal'
[http @ 0x102509500] (3671754)header='x-oss-hash-crc64ecma: 5079365695105828960'
[http @ 0x102509500] (3671754)header='x-oss-storage-class: Standard'
[http @ 0x102509500] (3671754)header='Content-MD5: le/erbhkgsVR8bz4aOzqQQ=='
[http @ 0x102509500] (3671754)header='x-oss-server-time: 33'
[http @ 0x102509500] (3671754)header='Via: cache31.l2nu20-2[0,200-0,H], cache10.l2nu20-2[2,0], cache7.cn307[0,200-0,H], cache3.cn307[28,0]'
[http @ 0x102509500] (3671754)header='Age: 763096'
[http @ 0x102509500] (3671754)header='X-Cache: HIT TCP_HIT dirn:5:256912714 mlen:-1'
[http @ 0x102509500] (3671754)header='X-Swift-SaveTime: Thu, 07 Dec 2017 15:58:06 GMT'
[http @ 0x102509500] (3671754)header='X-Swift-CacheTime: 2592000'
[http @ 0x102509500] (3671754)header='access-control-allow-origin: *'
[http @ 0x102509500] (3671754)header='Timing-Allow-Origin: *'
[http @ 0x102509500] (3671754)header='EagleId: 7b81d7cb15134159275121813e'
[http @ 0x102509500] (3671754)header=''
..........................................
.......................................... //下载1.ts文件

 http_connect
1 准备请求头：
先组装http请求，如："GET /otv/wxty/live/channel01/1300/20180129/20180129T152040.ts HTTP/1.1\r\nUser-Agent: Lavf/57.41.100\r\nAccept: */*\r\nConnection: close\r\nHost: gsports.livebak.ali.otvcloud.com\r\nIcy-MetaData: 1\r\n\r\n"
2 发送请求：
最终调用http的send（tcp.c的tcp_send）。
3 接收数据：
   /* wait for header */
    err = http_read_header(h, new_location);
此函数调用http_get_line，最终会调用tcp的read（tcp.c的tcp_read）, 得到tcp回复，数据存放在HTTPContext *s = h->priv_data; s为URLContext格式。

问题

1、M3u8是什么时候下载的？

答：在init_input->io_open的时候，请求下载

2、hls_probe是什么时候调用

答：init_input->av_probe_input_buffer2

3、m3u8是什么时候解析

答：hls_read_header

4、hls_read_packet什么时候调用

答：ffplay.c的av_read_frame

HLS官方网

Technical Note TN2288: Example Playlist Files for use with HTTP Live Streaming

ffmpeg 之 http 分析 一_陌上烟雨遥的博客-CSDN博客

ffmpeg 之 http 分析 二_陌上烟雨遥的博客-CSDN博客

 ffmpeg--http_evsqiezi-CSDN博客

Hls之交互流程_g0415shenw的博客-CSDN博客