2021-09-23 基于ffmpeg+nginx+rtsp的http-flv点播方案-推流失败问题分析
项目场景：
使用ffmpeg拉取rtsp服务器的码流，并且推rtmp流到nginx，web前端使用flv码流进行点播；
拉流以及推流使用的是ffmpeg原生接口；
点播服务器使用是NGINX，并集成了http-flv模块（可以参考我的其他文档。）
问题描述：
ffmpeg打开rtsp的url，从in_stream的码流中，复制所需要的h264编码信息；然后 写入对应的rtmp流（out_stream）中，结果发现写入函数av_interleaved_write_frame一直异常，无法写入？并且错误信息为“ Broke Pipe"? AVStream *in_stream = ifmt_ctx->streams[videoindex];
AVStream *out_stream = avformat_new_stream(ofmt_ctx, in_stream->codec->codec);
......
//复制AVCodecContext的设置（Copy the settings of AVCodecContext）
ret = avcodec_copy_context(out_stream->codec, in_stream->codec);
......
//紧接着调用打开、写入头、设置一些参数、循环读取数据、写入数据
ret = avio_open(&ofmt_ctx->pb, out_filename, AVIO_FLAG_WRITE);
......
ret = avformat_write_header(ofmt_ctx, /*NULL*/&options);
......
av_dump_format(ofmt_ctx, 0, out_filename, 1);
......
while (m_bRun)
{
AVStream *in_stream, *out_stream;
ret = av_read_frame(ifmt_ctx, &pkt);
if (ret < 0)
{
int err_code = ret;
char buf[1024] = {
0 };
if (ret < 0) {
av_strerror(err_code, buf, 1024);
qDebug()<<buf; //在此处报错
}
}
......
ret = av_interleaved_write_frame(ofmt_ctx, &pkt);
}
原因分析：
常规分析手段：-
抓包分析
通过抓包看，是nginx服务器主动结束了连接；
-
分析nginx日志
由于是在 av_interleaved_write_frame函数发生了异常，该函数直接作用是将“h264数据”送至nginx服务器，所以进一步打开nginx debug日志等级，错误日志如下：
这段日志中，明显可以看到的操作是在写 flv 的包头；举个例子，其中“onMetaData”等均是flv 包头格式；onMetaData是FLV文件中的第一个Tag, 用来表示当前文件的一些基本信息: 比如视音频的编码类型id、视频的宽和高、文件大小、视频长度、创建日期等。 顺着日志往下阅读，发现下面两行日志：2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 02 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0A ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (10) 6F 6E 4D 65 74 61 44 61 74 61 ‘onMetaData’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 03 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 06 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (6) 53 65 72 76 65 72 ‘Server’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 02 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 44 ‘?D’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (68) 4E 47 49 4E 58 20 48 54 54 50 2D 46 4C 56 20 28 ‘NGINX HTTP-FLV (’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 05 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (5) 77 69 64 74 68 ‘width’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 40 84 00 00 00 00 00 00 ‘@???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 06 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (6) 68 65 69 67 68 74 ‘height’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 40 7E 00 00 00 00 00 00 ‘@~???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0C ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (12) 64 69 73 70 6C 61 79 57 69 64 74 68 ‘displayWidth’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 40 84 00 00 00 00 00 00 ‘@???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0D ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (13) 64 69 73 70 6C 61 79 48 65 69 67 68 74 ‘displayHeight’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 40 7E 00 00 00 00 00 00 ‘@~???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 08 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 64 75 72 61 74 69 6F 6E ‘duration’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 09 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (9) 66 72 61 6D 65 72 61 74 65 ‘framerate’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 03 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (3) 66 70 73 ‘fps’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0D ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (13) 76 69 64 65 6F 64 61 74 61 72 61 74 65 ‘videodatarate’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0C ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (12) 76 69 64 65 6F 63 6F 64 65 63 69 64 ‘videocodecid’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 40 1C 00 00 00 00 00 00 ‘@???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0D ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (13) 61 75 64 69 6F 64 61 74 61 72 61 74 65 ‘audiodatarate’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 0C ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (12) 61 75 64 69 6F 63 6F 64 65 63 69 64 ‘audiocodecid’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 00 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (8) 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 07 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (7) 70 72 6F 66 69 6C 65 ‘profile’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 02 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 20 '? ’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (32) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 05 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (5) 6C 65 76 65 6C ‘level’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 02 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 20 '? ’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (32) 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ‘???’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (2) 00 00 ‘??’
2021/09/22 20:07:50 [debug] 1729538#0: *17 AMF write (1) 09 ‘?’
2021/09/22 20:07:50 [debug] 1729538#0: *17 RTMP prep amf_meta (18) fmt=0 csid=5 timestamp=0 mlen=409 msid=1 nbufs=4
2021/09/22 20:07:50 [debug] 1729538#0: *17 reusing formerly read data: 17
2021/09/22 20:07:50 [debug] 1729538#0: *17 RTMP bheader fmt=0 csid=6
2021/09/22 20:07:50 [debug] 1729538#0: *17 RTMP mheader fmt=0 video (9) time=0+0 mlen=5 len=0 msid=1
2021/09/22 20:07:50 [debug] 1729538#0: *17 RTMP recv video (9) csid=6 timestamp=0 mlen=5 msid=1 nbufs=1
2021/09/22 20:07:50 [debug] 1729538#0: *17 nhandlers: 6
2021/09/22 20:07:50 [debug] 1729538#0: *17 calling handler 0
2021/09/22 20:07:50 [debug] 1729538#0: *17 codec: avc header 1700000000
2021/09/22 20:07:50 [error] 1729538#0: *17 123456 codec: invalid video codec header size=5, client: 172.21.34.145, server: 0.0.0.0:1935
2021/09/22 20:07:50 [debug] 1729538#0: *17 handler 0 failed
2021/09/22 20:07:50 [debug] 1729538#0: *17 finalize session
2021/09/22 20:07:50 [debug] 1729538#0: *17 post event 0000000001654C48
2021/09/22 20:07:50 [debug] 1729538#0: timer delta: 42
2021/09/22 20:07:50 [debug] 1729538#0: posted event 0000000001654C48
2021/09/22 20:07:50 [debug] 1729538#0: *17 delete posted event 0000000001654C48
2021/09/22 20:07:50 [debug] 1729538#0: *17 close session
2021/09/22 20:07:50 [info] 1729538#0: *17 disconnect, client: 172.21.34.145, server: 0.0.0.0:1935
2021/09/22 20:07:50 [debug] 1729538#0: *17 codec: avc header 1700000000
2021/09/22 20:07:50 [error] 1729538#0: *17 123456 codec: invalid video codec header size=5, client: 172.21.34.145, server: 0.0.0.0:1935
avc header 是h264的编码头，这一段是无效数据，导致了异常，以及后面的close session;
- ffmpeg 函数对应的avc header是哪？
对应下面数据结构中的“ extradata”；
typedef struct AVCodecParameters {
/**
* General type of the encoded data.
*/
enum AVMediaType codec_type;
/**
* Specific type of the encoded data (the codec used).
*/
enum AVCodecID codec_id;
/**
* Additional information about the codec (corresponds to the AVI FOURCC).
*/
uint32_t codec_tag;
/**
* Extra binary data needed for initializing the decoder, codec-dependent.
*
* Must be allocated with av_malloc() and will be freed by
* avcodec_parameters_free(). The allocated size of extradata must be at
* least extradata_size + AV_INPUT_BUFFER_PADDING_SIZE, with the padding
* bytes zeroed.
*/
uint8_t *extradata;
/**
* Size of the extradata content in bytes.
*/
int extradata_size;
/**
* - video: the pixel format, the value corresponds to enum AVPixelFormat.
* - audio: the sample format, the value corresponds to enum AVSampleFormat.
*/
int format;
/**
* The average bitrate of the encoded data (in bits per second).
*/
int64_t bit_rate;
/**
* The number of bits per sample in the codedwords.
*
* This is basically the bitrate per sample. It is mandatory for a bunch of
* formats to actually decode them. It's the number of bits for one sample in
* the actual coded bitstream.
*
* This could be for example 4 for ADPCM
* For PCM formats this matches bits_per_raw_sample
* Can be 0
*/
int bits_per_coded_sample;
/**
* This is the number of valid bits in each output sample. If the
* sample format has more bits, the least significant bits are additional
* padding bits, which are always 0. Use right shifts to reduce the sample
* to its actual size. For example, audio formats with 24 bit samples will
* have bits_per_raw_sample set to 24, and format set to AV_SAMPLE_FMT_S32.
* To get the original sample use "(int32_t)sample >> 8"."
*
* For ADPCM this might be 12 or 16 or similar
* Can be 0
*/
int bits_per_raw_sample;
/**
* Codec-specific bitstream restrictions that the stream conforms to.
*/
int profile;
int level;
/**
* Video only. The dimensions of the video frame in pixels.
*/
int width;
int height;
/**
* Video only. The aspect ratio (width / height) which a single pixel
* should have when displayed.
*
* When the aspect ratio is unknown / undefined, the numerator should be
* set to 0 (the denominator may have any value).
*/
AVRational sample_aspect_ratio;
/**
* Video only. The order of the fields in interlaced video.
*/
enum AVFieldOrder field_order;
/**
* Video only. Additional colorspace characteristics.
*/
enum AVColorRange color_range;
enum AVColorPrimaries color_primaries;
enum AVColorTransferCharacteristic color_trc;
enum AVColorSpace color_space;
enum AVChromaLocation chroma_location;
/**
* Video only. Number of delayed frames.
*/
int video_delay;
/**
* Audio only. The channel layout bitmask. May be 0 if the channel layout is
* unknown or unspecified, otherwise the number of bits set must be equal to
* the channels field.
*/
uint64_t channel_layout;
/**
* Audio only. The number of audio channels.
*/
int channels;
/**
* Audio only. The number of audio samples per second.
*/
int sample_rate;
/**
* Audio only. The number of bytes per coded audio frame, required by some
* formats.
*
* Corresponds to nBlockAlign in WAVEFORMATEX.
*/
int block_align;
/**
* Audio only. Audio frame size, if known. Required by some formats to be static.
*/
int frame_size;
/**
* Audio only. The amount of padding (in samples) inserted by the encoder at
* the beginning of the audio. I.e. this number of leading decoded samples
* must be discarded by the caller to get the original audio without leading
* padding.
*/
int initial_padding;
/**
* Audio only. The amount of padding (in samples) appended by the encoder to
* the end of the audio. I.e. this number of decoded samples must be
* discarded by the caller from the end of the stream to get the original
* audio without any trailing padding.
*/
int trailing_padding;
/**
* Audio only. Number of samples to skip after a discontinuity.
*/
int seek_preroll;
} AVCodecParameters;
- 常规手段之，打印分析in_stream 以及out_stream的对应的parameter参数中的exteradata字段；
加入以下代码，将
void myprintf(uint8_t* data,int len)
{
QString disp_string,S;
QByteArray ba;
ba.resize(len);
QString m_strMSG = QString("myprintf len=:%1").arg(len);
for (int i = 0 ;i < len; i++)
{
ba[i] = data[i];
}
for(int i=0;i<ba.size();i++)
{
S.sprintf("0x%02x, ", (unsigned char)ba.at(i));
disp_string += S;
}
qDebug()<<m_strMSG<<"\n";
qDebug()<<disp_string<<"\n";
}
......
//进行打印
myprintf(out_stream->codecpar->extradata,out_stream->codecpar->extradata_size);
解决方案：
发现，从in_stream中获取的AVCodecParameters中的extradata是一串无效值；那么只需要将这个字段重新填入正确的值即可； 这段值应该填入正确的sps\pps等信息，剩下的就是编码实现了；举个例子，下面数据 17 00 00 00 00 就是video tag标签，后面圈出来的数据，就是对应的avc heaer信息，其中包含了正确的h264所需要的sps\pps等数据；
文章的最后：如果需要研究flv封装格式的特点，可以具体搜索网络资源。