ffmpeg h264实时流如何录像成Avi

h264一帧一帧的数据如何 赋值给av_interleaved_write_frame 函数进行写文件呢？

 

void :RawH264ToAvi(unsigned char *pH264Buff,int H264BuffLen)
{
 
 //
 AVPacket pkt;
 av_init_packet(&pkt);

 //
 pkt.flags        |= AV_PKT_FLAG_KEY;
 pkt.stream_index  = VideoDecodeSt->index;
 pkt.data          = pH264Buff;
 pkt.size          = H264BuffLen;/*sizeof(AVPicture)*/;

 pkt.pts=0;
 pkt.dts=0;

 int ret = av_interleaved_write_frame(avDecodeFmtContext, &pkt);
 

}

这样 录制的AVI用VLC播放正常，用风雷影音播放，发现帧率不能控制；

 

有一篇文章问题的问题和我一样：如下

http://stackoverflow.com/questions/5964142/raw-h264-frames-in-mpegts-container-using-libavcodec

http://stackoverflow.com/questions/1957427/detect-mpeg4-h264-i-frame-idr-in-rtp-stream

 

Raw H264 frames in mpegts container using libavcodec

 

I would really appreciate some help with the following issue:

I have a gadget with a camera, producing H264 compressed video frames, these frames are being sent to my application. These frames are not in a container, just raw data.

I want to use ffmpeg and libav functions to create a video file, which can be used later.

If I decode the frames, then encode them, everything works fine, I get a valid video file. (the decode/encode steps are the usual libav commands, nothing fancy here, I took them from the almighty internet, they are rock solid)... However, I waste a lot of time by decoding and encoding, so I would like to skip this step and directly put the frames in the output stream. Now, the problems come.

Here is the code I came up with for producing the encoding:

AVFrame* picture;

avpicture_fill((AVPicture*) picture, (uint8_t*)frameData, 
                 codecContext->pix_fmt, codecContext->width,
                 codecContext->height);
int outSize = avcodec_encode_video(codecContext, videoOutBuf, 
                 sizeof(videoOutBuf), picture);
if (outSize > 0) 
{
    AVPacket packet;
    av_init_packet(&packet);
    packet.pts = av_rescale_q(codecContext->coded_frame->pts,
                  codecContext->time_base, videoStream->time_base);
    if (codecContext->coded_frame->key_frame) 
    {
        packet.flags |= PKT_FLAG_KEY;
    }
    packet.stream_index = videoStream->index;
    packet.data =  videoOutBuf;
    packet.size =  outSize;

    av_interleaved_write_frame(context, &packet);
    put_flush_packet(context->pb);
}

Where the variables are like:

frameData is the decoded frame data, that came from the camera, it was decoded in a previous step andvideoOutBuf is a plain uint8_t buffer for holding the data

I have modified the application in order to not to decode the frames, but simply pass through the data like:

    AVPacket packet;
    av_init_packet(&packet);

    packet.stream_index = videoStream->index;
    packet.data = (uint8_t*)frameData;
    packet.size = currentFrameSize;

    av_interleaved_write_frame(context, &packet);
    put_flush_packet(context->pb);

where

frameData is the raw H264 frame andcurrentFrameSize is the size of the raw H264 frame, ie. the number of bytes I get from the gadget for every frame.

And suddenly the application is not working correctly anymore, the produced video is unplayable. This is obvious, since I was not setting a correct PTS for the packet. What I did was the following (I'm desperate, you can see it from this approach :) )

    packet.pts = timestamps[timestamp_counter ++];

where timestamps is actually a list of PTS's produced by the working code above, and written to a file (yes, you read it properly, I logged all the PTS's for a 10 minute session, and wanted to use them).

The application still does not work.

Now, here I am without any clue what to do, so here is the question:

I would like to create an "mpegts" video stream using libav functions, insert in the stream already encoded video frames and create a video file with it. How do I do it?

Thanks, f.

 

有人回答：

I believe if you set the following, you will see video playback.

packet.flags |= AV_PKT_FLAG_KEY;
packet.pts = packet.dts = 0;

You should really set packet.flags according to the h264 packet headers. You might trythis fellow stack overflowian's suggestion for extracting directly from the stream.

If you are also adding audio, then pts/dts is going to be more important. I suggest you studythis tutorial

EDIT

I found time to extract out what is working for me from my test app. For some reason, dts/pts values of zero works for me, but values other than 0 or AV_NOPTS_VALUE do not. I wonder if we have different versions of ffmpeg. I have the latest fromgit://git.videolan.org/ffmpeg.git.

 

代码：

#define UINT64_C(c) (c ## ULL)
#endif

//#define _M
#define _M printf( "%s(%d) : MARKER\n", __FILE__, __LINE__ )

extern "C"
{
    #include "libavcodec/avcodec.h"
    #include "libavformat/avformat.h"
};

AVFormatContext *fc = 0;
int vi = -1, waitkey = 1;

// < 0 = error
// 0 = I-Frame
// 1 = P-Frame
// 2 = B-Frame
// 3 = S-Frame
int getVopType( const void *p, int len )
{  
    if ( !p || 6 >= len )
        return -1;

    unsigned char *b = (unsigned char*)p;

    // Verify NAL marker
    if ( b[ 0 ] || b[ 1 ] || 0x01 != b[ 2 ] )
    {   b++;
        if ( b[ 0 ] || b[ 1 ] || 0x01 != b[ 2 ] )
            return -1;
    } // end if

    b += 3;

    // Verify VOP id
    if ( 0xb6 == *b )
    {   b++;
        return ( *b & 0xc0 ) >> 6;
    } // end if

    switch( *b )
    {   case 0x65 : return 0;
        case 0x61 : return 1;
        case 0x01 : return 2;
    } // end switch

    return -1;
}

void write_frame( const void* p, int len )
{
    if ( 0 > vi )
        return;

    AVStream *pst = fc->streams[ vi ];

    // Init packet
    AVPacket pkt;
    av_init_packet( &pkt );
    pkt.flags |= ( 0 >= getVopType( p, len ) ) ? AV_PKT_FLAG_KEY : 0;  
    pkt.stream_index = pst->index;
    pkt.data = (uint8_t*)p;
    pkt.size = len;

    // Wait for key frame
    if ( waitkey )
        if ( 0 == ( pkt.flags & AV_PKT_FLAG_KEY ) )
            return;
        else
            waitkey = 0;

    pkt.dts = AV_NOPTS_VALUE;
    pkt.pts = AV_NOPTS_VALUE;

//  av_write_frame( fc, &pkt );
    av_interleaved_write_frame( fc, &pkt );
}

void destroy()
{
    waitkey = 1;
    vi = -1;

    if ( !fc )
        return;

_M; av_write_trailer( fc );

    if ( fc->oformat && !( fc->oformat->flags & AVFMT_NOFILE ) && fc->pb )
        avio_close( fc->pb );

    // Free the stream
_M; av_free( fc );

    fc = 0;
_M;
}

int get_nal_type( void *p, int len )
{
    if ( !p || 5 >= len )
        return -1;

    unsigned char *b = (unsigned char*)p;

    // Verify NAL marker
    if ( b[ 0 ] || b[ 1 ] || 0x01 != b[ 2 ] )
    {   b++;
        if ( b[ 0 ] || b[ 1 ] || 0x01 != b[ 2 ] )
            return -1;
    } // end if

    b += 3;

    return *b;
}

int create( void *p, int len )
{
    if ( 0x67 != get_nal_type( p, len ) )
        return -1;

    destroy();

    const char *file = "test.avi";
    CodecID codec_id = CODEC_ID_H264;
//  CodecID codec_id = CODEC_ID_MPEG4;
    int br = 1000000;
    int w = 480;
    int h = 354;
    int fps = 15;

    // Create container
_M; AVOutputFormat *of = av_guess_format( 0, file, 0 );
    fc = avformat_alloc_context();
    fc->oformat = of;
    strcpy( fc->filename, file );

    // Add video stream
_M; AVStream *pst = av_new_stream( fc, 0 );
    vi = pst->index;

    AVCodecContext *pcc = pst->codec;
_M; avcodec_get_context_defaults2( pcc, AVMEDIA_TYPE_VIDEO );
    pcc->codec_type = AVMEDIA_TYPE_VIDEO;

    pcc->codec_id = codec_id;
    pcc->bit_rate = br;
    pcc->width = w;
    pcc->height = h;
    pcc->time_base.num = 1;
    pcc->time_base.den = fps;

    // Init container
_M; av_set_parameters( fc, 0 );

    if ( !( fc->oformat->flags & AVFMT_NOFILE ) )
        avio_open( &fc->pb, fc->filename, URL_WRONLY );

_M; av_write_header( fc );

_M; return 1;
}

int main( int argc, char** argv )
{
    int f = 0, sz = 0;
    char fname[ 256 ] = { 0 };
    char buf[ 128 * 1024 ];

    av_log_set_level( AV_LOG_ERROR );
    av_register_all();

    do
    {
        // Raw frames in v0.raw, v1.raw, v2.raw, ...
//      sprintf( fname, "rawvideo/v%lu.raw", f++ );
        sprintf( fname, "frames/frame%lu.bin", f++ );
        printf( "%s\n", fname );

        FILE *fd = fopen( fname, "rb" );
        if ( !fd )
            sz = 0;
        else
        {
            sz = fread( buf, 1, sizeof( buf ) - FF_INPUT_BUFFER_PADDING_SIZE, fd );
            if ( 0 < sz )
            {
                memset( &buf[ sz ], 0, FF_INPUT_BUFFER_PADDING_SIZE );         

                if ( !fc )
                    create( buf, sz );

                if ( fc )
                    write_frame( buf, sz );

            } // end if

            fclose( fd );

        } // end else

    } while ( 0 < sz );

    destroy();
}

 

////////////////////////////////////////////////////////////////////////////////////////////////////

Ok so I figured it out for h264 stream.

How to detect I-Frame:

• remove RTP header
• check the value of the first byte in h264 payload
• if the value is 124 (0x7C) it is an I-Frame

I cant figure it out for the MPEG4-ES stream... any suggestions?

EDIT: H264 IDR

This works for my h264 stream (fmtp:96 packetization-mode=1; profile-level-id=420029;). You just pass byte array that represents the h264 fragment received through RTP. If you want to pass whole RTP, just correct theRTPHeaderBytes value to skip RTP header. I always get the I-Frame, because it is the only frame that can be fragmented, seehere. I use this (simplified) piece of code in my server, and it works like a charm!!!! If the I-Frame (IDR) is not fragmented, thefragment_type would be 5, so this code would return true for the fragmented and not fragmented IDRs.

 

 

public static bool isH264iFrame(byte[] paket)
    {
        int RTPHeaderBytes = 0;

        int fragment_type = paket[RTPHeaderBytes + 0] & 0x1F;
        int nal_type = paket[RTPHeaderBytes + 1] & 0x1F;
        int start_bit = paket[RTPHeaderBytes + 1] & 0x80;

        if (((fragment_type == 28 || fragment_type == 29) && nal_type == 5 && start_bit == 128) || fragment_type == 5)
        {
            return true;
        }

        return false;
   }

 

Here's the table of NAL unit types:

 Type Name
    0 [unspecified]
    1 Coded slice
    2 Data Partition A
    3 Data Partition B
    4 Data Partition C
    5 IDR (Instantaneous Decoding Refresh) Picture
    6 SEI (Supplemental Enhancement Information)
    7 SPS (Sequence Parameter Set)
    8 PPS (Picture Parameter Set)
    9 Access Unit Delimiter
   10 EoS (End of Sequence)
   11 EoS (End of Stream)
   12 Filter Data
13-23 [extended]
24-31 [unspecified] 

EDIT 2: MPEG4 I-VOP

I forgot to update this... Thanx to Che and ISO IEC 14496-2 document, I managed to work this out! Che was rite, but not so precise in his answer... so here is how to find I, P and B frames (I-VOP, P-VOP, B-VOP) in short:

1. VOP (Video Object Plane -- frame) starts with a code 000001B6(hex). It is the same for all MPEG4 frames (I,P,B)

2. Next follows many more info, that I am not going to describe here (see the IEC doc), but we only (as che said) need the higher 2 bits from the following byte (next two bits after the byte with the value B6). Those 2 bits tell you the VOP_CODING_TYPE, see the table:

   VOP_CODING_TYPE (binary)  Coding method
                         00  intra-coded (I)
                         01  predictive-coded (P)
                         10  bidirectionally-predictive-coded (B)
                         11  sprite (S)
   

So, to find I-Frame find the packet starting with four bytes 000001B6 and having the higher two bits of the next byte 00. This will find I frame in MPEG4 stream with a simple video object type (not sure for advanced simple).

For any other problems, you can check the document provided (ISO IEC 14496-2), there is all you want to know about MPEG4. :)