With the release of the iPhone OS 3 update came the ability to do live streaming. There are a few types of streaming and each requires a certain encoding and segmentation. I've put together a cheat sheet on how I went about building a static stream using FFMpeg and an example segmenter that someone has posted. I'm not covering windowed streams in this post but if you are thinking about implementing a windowed stream the following will help you make a step in that direction.
Before getting started it is best to read over the Apple documentation on HTTP live streaming. Start out with the iPhone streaming media overview. This document covers the basics of how the streaming works and has some nice diagrams.
If you want even more information after reading the overview you can take a look at the HTTP Live streaming draft proposal that was submitted to the IETF by Apple. It covers the streaming protocol in complete detail and has examples of the stream file format for reference.
Once you are ready to start grab a decent quality video clip to use. If you don't have one handy I found a nice list of downloadable HD clips in various formats for testing.
Step 1: Grab the latest version of FFMpeg
You may be able to get away with anything after FFMpeg 0.5 but you might as well pull down a more recent version. The FFMpeg download page has instructions on getting the latest version. I pulled the version I used out of git.
I used the following command to configure FFMpeg:
configure --enable-gpl --enable-nonfree --enable-pthreads --enable-libfaac --enable-libfaad --enable-libmp3lame --enable-libx264
One of the main things to note is the --enable-libx264 flag.
Step 2: Encode your video for the iPhone
Once you have a working version of FFMpeg it is time to create an X264 encoded stream that will work with the iPhone. There are a few things to note before diving in:
- The supported bitrates for streaming are: 100 Kbps to 1.6 Mbps
- The suggested bitrates for streaming are*:
- Low – 96 Kbps video, 64 Kbps audio
- Medium – 256 Kbps video, 64 Kbps audio
- High – 800 Kbps video, 64 Kbps audio
- The iPhone screen size is: 480×320
* See step 7 for more information on what I think are better bitrates.
Taking all that into account someone on the iPhone developer forums suggested the following and it works well for me:
If you want some more detail on some of these commands check out the X264 encoding guide and in general the FFMpeg documentation to see what all the flags mean.
Note that I have the bitrate set to 96k in the above example. That can be changed to fit your needs. Use the script that I have created later in the post or just make sure you change the -b, -maxrate, and -bufsize values.
Step 3: Download and build the segmenter
Now you have a complete video but you don't want to toss the entire thing up or you wouldn't be reading about HTTP streaming. What you need is a way to segment the video stream into smaller chunks. You can download Apple's segmenter (see the overview above for more information on where to find it) or you can download one created by the forum user corp186.
There is an SVN repository set up for the segmenter source. It is only a couple files and it is easy to build. The trouble you may run into is that the Makefile that it comes with won't build the binary correctly. Don't worry it just takes some extra link flags to make it work. The following is what I needed in the Makefile to get it to build on my system:
gcc -Wall -g segmenter.c -o segmenter -lavformat -lavcodec -lavutil -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad
clean:
rm segmenter
After compiling the segmenter you are ready to create your first HTTP streaming content.
The format of the segmenter command is:
Following is an example used to create a stream from a video file created with the above FFMpeg command split into 10 second intervals:
Step 4: Prepare the HTTP server
At this point you should have a set of files that represent the stream and a stream definition file. Those files can be uploaded to a web server at this point but there is another important step to take that ensures they will be download correctly and that is setting up mime types. There are two mime types that are important for the streaming content:
.ts video/MP2T
If you are using Apache you would want to add the following to your httpd.conf file:
AddType video/MP2T .ts
If you are using lighttpd you would want to put this in your configuration file (if you have other mime types defined make sure you just add these and don't set them):
Step 5: Test the stream
The video is encoded for the iPhone, segmented for streaming, and the server is configured. The only thing left to do is test the stream and the fastest way to do that is to use the new HTML5 video tag. Here is an example of how to set it up:
<head>
<title>Video Test </title>
<meta name="viewport" content="width=320; initial-scale=1.0; maximum-scale=1.0; user-scalable=0;"/>
</head>
<body style="background-color:#FFFFFF; ">
<center>
<video width='150' height='150' src="stream-128k.m3u8" />
</center>
</body>
</html>
If everything has been done correctly you should see the video.
If you want to test the stream out in an application then download the MoviePlayer iPhone demo application from the iPhone developer site. Build and run it in the simulator or put it on an actual phone and then type the URL in for the server you uploaded your stream to.
That is all there is to building a single static HTTP stream. A good number of steps but if you have some experience using FFMpeg it isn't too hard to set up. The only pitfalls I ran into revolve around trying to segment the stream without the segmeter code. I don't know enough about how the segmentation works to know why this is so difficult to do but I believe it could have something to do with synchronization points in the stream. Of course when you stray from the path the stream just doesn't work and you get a generic error message so that is just my best guess. I'll also guess that Apple may tighten up the player over time and make it work better with miss-formatted streams.
Step 6: Automating the stream encoding and segmentation
Here is a little script I put together that first encodes an input file and then segments it into 10 second chunks:
BR=800k
ffmpeg -i $1 -f mpegts -acodec libmp3lame -ar 48000 -ab 64k -s320×240 -vcodec libx264 -b $BR -flags +loop -cmp +chroma -partitions +parti4x4+partp8x8+partb8x8 -subq 5 -trellis 1 -refs1 -coder 0 -me_range 16 -keyint_min 25 -sc_threshold 40 -i_qfactor 0.71 -bt 200k -maxrate $BR -bufsize $BR -rc_eq'blurCplx^(1-qComp)' -qcomp 0.6 -qmin 10 -qmax 51 -qdiff 4 -level 30 -aspect 320:240 -g 30 -async 2 sample_$BR_pre.ts
segmenter sample_$BR_pre.ts 10 sample_$BR stream-$BR.m3u8 http://www.ioncannon.net/
rm -f sample_$BR_pre.ts
The script could use some work but it does a good enough job for testing.
Step 7: Create a variable rate HTTP stream
Once you have creating a single stream down you need to try out creating a variable bitrate stream. There isn't much to it, just create different bitrate encoded streams and link to their stream definition files in a separate stream definition file. Here is an example:
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=96000
http://192.168.132.15/ipv/stream-96k.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=256000
http://192.168.132.15/ipv/stream-256k.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=800000
http://192.168.132.15/ipv/stream-800k.m3u8
I gave the above a try using both the iPhone's 3G connection and a WIFI connection. The following log shows the two different attempts (first 3G then WIFI):
6.x.x.x ioncannon.net - [20:49:13] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..." 6.x.x.x ioncannon.net - [20:49:14] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..." 6.x.x.x ioncannon.net - [20:49:15] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..." 6.x.x.x ioncannon.net - [20:49:16] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..." 6.x.x.x ioncannon.net - [20:49:18] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..." 6.x.x.x ioncannon.net - [20:49:21] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..." 6.x.x.x ioncannon.net - [20:49:30] "GET /s_96k-00003.ts HTTP/1.1" 200 383520 "-" "..." 6.x.x.x ioncannon.net - [20:49:32] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..." 6.x.x.x ioncannon.net - [20:49:39] "GET /s_256k-00003.ts HTTP/1.1" 200 716844 "-" "..." 6.x.x.x ioncannon.net - [20:49:49] "GET /s_256k-00004.ts HTTP/1.1" 200 705564 "-" "..." 6.x.x.x ioncannon.net - [20:49:57] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..." 6.x.x.x ioncannon.net - [20:49:59] "GET /s_96k-00004.ts HTTP/1.1" 200 368668 "-" "..." 6.x.x.x ioncannon.net - [20:50:03] "GET /s_96k-00005.ts HTTP/1.1" 200 371300 "-" "..." 6.x.x.x ioncannon.net - [20:50:13] "GET /s_96k-00006.ts HTTP/1.1" 200 398936 "-" "..." 6.x.x.x ioncannon.net - [20:50:16] "GET /stream-256k.m3u8 HTTP/1.1" 200 730 "-" "..." 6.x.x.x ioncannon.net - [20:50:22] "GET /s_256k-00006.ts HTTP/1.1" 200 758016 "-" "..." 6.x.x.x ioncannon.net - [20:50:36] "GET /s_256k-00007.ts HTTP/1.1" 200 737524 "-" "..." 6.x.x.x ioncannon.net - [20:50:40] "GET /s_256k-00008.ts HTTP/1.1" 200 773244 "-" "..." 6.x.x.x ioncannon.net - [20:50:46] "GET /s_256k-00009.ts HTTP/1.1" 200 717032 "-" "..." 6.x.x.x ioncannon.net - [20:50:57] "GET /s_256k-00010.ts HTTP/1.1" 200 768920 "-" "..." 6.x.x.x ioncannon.net - [20:51:06] "GET /s_256k-00011.ts HTTP/1.1" 200 611000 "-" "..." 1.x.x.x ioncannon.net - [20:52:23] "GET /varpl.m3u8 HTTP/1.1" 304 0 "-" "..." 1.x.x.x ioncannon.net - [20:52:24] "GET /varpl.m3u8 HTTP/1.1" 206 288 "-" "..." 1.x.x.x ioncannon.net - [20:52:25] "GET /varpl.m3u8 HTTP/1.1" 200 288 "-" "..." 1.x.x.x ioncannon.net - [20:52:25] "GET /stream-96k.m3u8 HTTP/1.1" 200 719 "-" "..." 1.x.x.x ioncannon.net - [20:52:26] "GET /s_96k-00001.ts HTTP/1.1" 200 334828 "-" "..." 1.x.x.x ioncannon.net - [20:52:27] "GET /s_96k-00002.ts HTTP/1.1" 200 377880 "-" "..." 1.x.x.x ioncannon.net - [20:52:28] "GET /stream-800k.m3u8 HTTP/1.1" 200 730 "-" "..." 1.x.x.x ioncannon.net - [20:52:31] "GET /s_800k-00002.ts HTTP/1.1" 200 1774156 "-" "..." 1.x.x.x ioncannon.net - [20:52:34] "GET /s_800k-00003.ts HTTP/1.1" 200 1916096 "-" "..." 1.x.x.x ioncannon.net - [20:52:38] "GET /s_800k-00004.ts HTTP/1.1" 200 1831872 "-" "..." 1.x.x.x ioncannon.net - [20:52:41] "GET /s_800k-00005.ts HTTP/1.1" 200 1831496 "-" "..." 1.x.x.x ioncannon.net - [20:52:46] "GET /s_800k-00006.ts HTTP/1.1" 200 1967608 "-" "..." 1.x.x.x ioncannon.net - [20:52:50] "GET /s_800k-00007.ts HTTP/1.1" 200 1676208 "-" "..." 1.x.x.x ioncannon.net - [20:52:54] "GET /s_800k-00008.ts HTTP/1.1" 200 2094132 "-" "..." 1.x.x.x ioncannon.net - [20:52:58] "GET /s_800k-00009.ts HTTP/1.1" 200 1860260 "-" "..." 1.x.x.x ioncannon.net - [20:53:08] "GET /s_800k-00010.ts HTTP/1.1" 200 2008404 "-" "..." 1.x.x.x ioncannon.net - [20:53:19] "GET /s_800k-00011.ts HTTP/1.1" 200 1400224 "-" "..."
Notice that there is a decent bit of indecisiveness on the part of what stream to pick when using 3G. For my test it actually caused the player to pause while it switched from the 256k stream back to the 96k stream. The stream on the WIFI connection starts out low but then jumps right to the highest quality and stays there. Overall it seems like the variable rate streaming works decently and again Apple may be able to tweak it down the road to get even better results.
The bitrate jump between 96k and 256k is probably too large even though that is what Apple seems to recommend. I believe with some testing a better set of bitrates could be found. The video quality of the 256k bitrate looks pretty good so I would say that 96k, 128k and 384k would potentially be a better choice.
Some parting notes
If you are interested in how the segmenter works you can find out more on how to use libavformat at the following resources: an older libavformat tutorial, some sample libavformat code, How to Write a Video Player in Less Than 1000 Lines, and more sample libavformat code.
The next step for this is to do a windowed live stream. I've done a little experimenting so far and with a modified segmeter I can generate a live stream. I will need to heavily modify the segmeter to get a live windowed stream so it may take a little while to get it done. My intent of course will be to combine the modifications with something fun like S3 and cloudfront since I believe that would be a sweat combination.
188 Comments
I'm trying to build the segmenter on Ubuntu 10.04, like crazyfool2100 apparently did successfully, and just cannot do it to save my life. I've followed all instructions, and still get a string of "undefined reference" errors, starting with:
gcc -Wall -I/home/jkb/ffmpeg -g segmenter.c -o segmenter -L/home/jkb/ffmpeg/libavformat -lavformat -L/home/jkb/ffmpeg/libavcodec -lavcodec -L/home/jkb/ffmpeg/libavutil -lavutil -I/home/jkb/ffmpeg -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -lpthread
/home/jkb/ffmpeg/libavcodec/libavcodec.a(bink.o): In function `decode_init':
/home/jkb/ffmpeg/libavcodec/bink.c:975: undefined reference to `av_image_check_size'
I was able to build ffmpeg fine, and run it to create an .mp4 file. But I get no further because I can't segment it.
Does somebody just have a binary of it???
Hey,
I got this little issue about duration and can't find a solution. Everything works fine, video result looks create on iphone and ipad but the time is always wrong so video stops and audio goes on for some seconds. In my logs I found:
E, [2010-09-09T14:23:32.894933 #28401] ERROR — : Encoder c440k: [mpegts @ 0x8053150] max_analyze_duration reached
[mpegts @ 0x8053150] Estimating duration from bitrate, this may be inaccurate
Output #0, mpegts, to 'MOV_9581431085c867db3e78223a407bb77b334417dc_c440k':
Stream #0.0: Video: libx264, yuv420p, 400×224, q=2-31, 90k tbn, 30 tbc
Stream #0.1: Audio: libfaac, 44100 Hz, stereo, 63 kb/s
[mpegts @ 0x8060450] muxrate VBR, pcr every 6 pkts, sdt every 200, pat/pmt every 40 pkts
[mpegts @ 0x8060450] st:1 error, non monotone timestamps 126000 >= 126000
Segmenter error: Could not write frame of stream: -1
frame= 210 fps= 37 q=29.0 size= 359kB time=5.63 bitrate= 522.2kbits/s
Tried with all kind of video input and the recommended ipad-config.yml
Thanks for any ideas.
This is what seems to work for me to compile the segmenter. Remember to change the path for each library.
gcc -Wall -g segmenter.c -o segmenter -L/usr/include/ffmpeg/libavformat -lavformat -L/usr/include/ffmpeg/libavcodec -lavcodec -L/usr/include/ffmpeg/libavutil -lavutil -I/usr/include/ffmpeg/ -lm -lmp3lame -lxvidcore -lx264 -lfaad -lfaac -lpthread -lz
Greetings!
Can segmenter be statically linked to .a libraries instead of .so?
Has anyone tried and succeeded?
Hi, I encounter same problem compiling segmenter that JkB
/usr/local/lib/libavcodec.a(bink.o): In function `decode_init':
/root/ffmpeg/libavcodec/bink.c:975: undefined reference to `av_image_check_size'
/usr/local/lib/libavcodec.a(dnxhddec.o): In function `dnxhd_decode_frame':
/root/ffmpeg/libavcodec/dnxhddec.c:309: undefined reference to `av_image_check_size'
…
I've undertstood that problem is coming from ffmpeg revision …
I've tried to set Paths in Makefile … but problem is still there.
Does somebody find a solution or have a clue to fix this ?
Thanks for any good idea
Hi, I am also having problem in compiling segmenter.
I tried following comment:
gcc -Wall -g live_segmenter.c -o live_segmenter -L/root/ffmpeg/libavformat -lavformat -L/root/ffmpeg/libavcodec -lavcodec -L/root/ffmpeg/libavutil -lavutil -I/root/ffmpeg/ -lm -lmp3lame -lxvidcore -lx264 -lfaad -lfaac -lpthread -lz
And I got following errors.
/root/ffmpeg/libavcodec/libavcodec.a(bink.o): In function `decode_init':
/root/ffmpeg/libavcodec/bink.c:975: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(dnxhddec.o): In function `dnxhd_decode_frame':
/root/ffmpeg/libavcodec/dnxhddec.c:309: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(dpx.o): In function `decode_frame':
/root/ffmpeg/libavcodec/dpx.c:143: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(dsputil.o): In function `dsputil_init':
/root/ffmpeg/libavcodec/dsputil.c:4458: undefined reference to `av_image_copy_plane'
/root/ffmpeg/libavcodec/libavcodec.a(eacmv.o): In function `cmv_decode_frame':
/root/ffmpeg/libavcodec/eacmv.c:160: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(eamad.o): In function `decode_frame':
/root/ffmpeg/libavcodec/eamad.c:264: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(eatgv.o): In function `tgv_decode_frame':
/root/ffmpeg/libavcodec/eatgv.c:279: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(gifdec.o): In function `gif_decode_frame':
/root/ffmpeg/libavcodec/gifdec.c:300: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(h264.o): In function `decode_slice_header':
/root/ffmpeg/libavcodec/h264.c:1918: undefined reference to `av_image_copy'
/root/ffmpeg/libavcodec/libavcodec.a(iff.o): In function `decode_init':
/root/ffmpeg/libavcodec/iff.c:164: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(indeo3.o): In function `iv_decode_frame':
/root/ffmpeg/libavcodec/indeo3.c:999: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(kgv1dec.o): In function `decode_frame':
/root/ffmpeg/libavcodec/kgv1dec.c:54: undefined reference to `av_image_check_size'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_wb_decode_frame':
/root/ffmpeg/libavcodec/libopencore-amr.c:300: undefined reference to `D_IF_decode'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_wb_decode_close':
/root/ffmpeg/libavcodec/libopencore-amr.c:309: undefined reference to `D_IF_exit'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_nb_encode_frame':
/root/ffmpeg/libavcodec/libopencore-amr.c:208: undefined reference to `Encoder_Interface_Encode'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_nb_decode_frame':
/root/ffmpeg/libavcodec/libopencore-amr.c:136: undefined reference to `Decoder_Interface_Decode'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_wb_decode_init':
/root/ffmpeg/libavcodec/libopencore-amr.c:263: undefined reference to `D_IF_init'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_nb_encode_close':
/root/ffmpeg/libavcodec/libopencore-amr.c:191: undefined reference to `Encoder_Interface_exit'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_nb_encode_init':
/root/ffmpeg/libavcodec/libopencore-amr.c:173: undefined reference to `Encoder_Interface_init'
/root/ffmpeg/libavcodec/libavcodec.a(libopencore-amr.o): In function `amr_nb_decode_close':
/root/ffmpeg/libavcodec/libopencore-amr.c:105: undefined reference to `Decoder_Interface_exit'
…….
…….. and many more…….
……..
………
/root/ffmpeg/libavcodec/libavcodec.a(imgconvert.o): In function `ff_fill_linesize':
/root/ffmpeg/libavcodec/imgconvert.c:504: undefined reference to `av_image_fill_linesizes'
/root/ffmpeg/libavcodec/libavcodec.a(imgconvert.o): In function `avpicture_fill':
/root/ffmpeg/libavcodec/imgconvert.c:524: undefined reference to `av_image_fill_pointers'
collect2: ld returned 1 exit status
make: *** [all] Error 1
Can anyone please suggest any solution to get rid of all the errors above and can successfully install segmenter.
Any help is really appreciable…..
Thanks in advance…
I am also getting similar error as Hervé and JkB.
Trying to build segmenter on Ubuntu 10.04, 32 bit machine.
I've installed ffmpeg as been described in ubuntu documentation, but still getting errors .
Does anyone have any solution or fix for this?
Thanks in advance.
I'm working on building a video uploading app for work and I can't seem to get this segmentation to work perfectly. I'm encoding the videos using ffmpeg and these settings:
c:\ffmpeg\bin\ffmpeg.exe -i c:\ffmpeg\bin\Input\test.avi -vcodec libx264 -f mpegts -vpre slower -vpre baseline -b 160k -r 10 -g 25 -pass 1/2 -threads 8 -y -strict experimental -acodec aac -ar 48000 -ab 64k 160k_stream.ts
c:\ffmpeg\bin\ffmpeg.exe -i c:\ffmpeg\bin\Input\test.avi -vcodec libx264 -f mpegts -vpre slower -vpre baseline -b 300k -r 10 -g 25 -pass 1/2 -threads 8 -y -strict experimental -acodec aac -ar 48000 -ab 64k sample_300_pre.ts
And to get an audio only track:
c:\ffmpeg\bin\ffmpeg.exe -i c:\ffmpeg\bin\output\300k_stream.ts -acodec copy -vn -y output/audio_stream.ts
Everything in that department works great. The videos play just fine. The problem comes when I run the segmenter:
c:\ffmpeg\bin\segmenter.exe c:\ffmpeg\bin\output\audio_stream.ts 10 stream c:/ffmpeg/bin/segmented/stream_audio.m3u8 audio\
c:\ffmpeg\bin\segmenter.exe c:\ffmpeg\bin\output\160k_stream.ts 10 stream c:/ffmpeg/bin/segmented/stream_low.m3u8 low\
c:\ffmpeg\bin\segmenter.exe c:\ffmpeg\bin\output\300k_stream.ts 10 stream c:/ffmpeg/bin/segmented/stream_medium.m3u8 medium\
It seems the segmenter never seems to cut the files into the same number of pieces. Moreover, the pieces are of different lengths, not 10 seconds each like I originally thought they would be. (piece 1 on low may be 9 seconds where it is 11 seconds on medium and 10 seconds on audio only).
Needless to say, I'm stumped. Is there any way to force some kind of synchronization here? This is so I can do real live streaming with intelligent bitrate switching.
@Reini You won't always get an exact 10 second segment and that is acceptable. It should be close to 10 seconds each time but it can be over or under. Each segment has to stop and start on a special frame so it may be that for the different qualities ffmpeg is adding those frames in different places.
For those having trouble with the video playing twice as fast as it should and the audio being off, these are the changes I had to make to get it working. There are also a couple small updates so it shouldn't complain about newer ffmpeg versions, and ignoring the warning about DTS/PTS values (as suggested by John).
This version ignores any initial audio or video delay values (I'm assuming both streams *should* start at zero, which is the case with our videos). Using this method seems to solve at least our issues with "non monotone timestamps" as well as fixes the sync problems we've seen.
The main trick about the frame rate is that newer ffmpeg versions set a ticks_per_frame multiplier (of 2) on the time base, but this isn't accounted for in the segmenter. So I added something to take that into account.
I'm not sure these changes will work for everyone, but they fix the issues we've run into. I'd be interested to hear any comments on the correctness of these changes.
Oh, and one more note for those having trouble compiling, getting errors starting with "/root/ffmpeg/libavcodec/libavcodec.a(bink.o): In function `decode_init':" — you probably have ffmpeg (and libavcodec) compiled statically. Try compiling ffmpeg with –enable-shared and maybe –disable-static to see if it helps?
===================================================================
— segmenter.c (revision 50)
+++ segmenter.c (working copy)
@@ -239,7 +239,7 @@
exit(1);
}
- ofmt = guess_format("mpegts", NULL, NULL);
+ ofmt = av_guess_format("mpegts", NULL, NULL);
if (!ofmt) {
fprintf(stderr, "Could not find MPEG-TS muxer\n");
exit(1);
@@ -251,6 +251,9 @@
exit(1);
}
oc->oformat = ofmt;
+
+ // Don't print warnings when PTS and DTS are identical.
+ ic->flags |= AVFMT_FLAG_IGNDTS;
video_index = -1;
audio_index = -1;
@@ -289,6 +292,12 @@
fprintf(stderr, "Could not open video decoder, key frames will not be honored\n");
}
+ if (video_st->codec->ticks_per_frame > 1) {
+ // h264 sets the ticks_per_frame and time_base.den but not time_base.num
+ // since we don't use ticks_per_frame, adjust time_base.num accordingly.
+ video_st->codec->time_base.num *= video_st->codec->ticks_per_frame;
+ }
+
snprintf(output_filename, strlen(output_prefix) + 15, "%s-%u.ts", output_prefix, output_index++);
if (url_fopen(&oc->pb, output_filename, URL_WRONLY) pts.val * video_st->time_base.num / video_st->time_base.den;
- }
- else if (video_index pts.val * audio_st->time_base.num / audio_st->time_base.den;
- }
- else {
+ if (packet.stream_index == video_index) {
+ if (initial_video_pts time_base.num / video_st->time_base.den;
+ } else {
+ segment_time = prev_segment_time;
+ }
+ } else if (packet.stream_index == audio_index) {
+ if (initial_audio_pts = segment_duration) {
@@ -357,7 +378,7 @@
prev_segment_time = segment_time;
}
- ret = av_interleaved_write_frame(oc, &packet);
+ ret = av_write_frame(oc, &packet);
if (ret < 0) {
fprintf(stderr, "Warning: Could not write frame of stream\n");
}
Hey Carson,
Excellent post! I learned a lot for sure. One question: I noticed on another site that there is a way to push a stream via UDP.http://ubuntuforums.org/showthread.php?p=7257944 I was wondering how something like this would fit in your scenario. Could I be sending a file through FFMPEG, creating a live stream in real time?
A couple people have emailed, mentioning that the blog reformatted my patch so that it's difficult to use. Here's a gist containing the same patch:https://gist.github.com/776283
@Justin: Thanks, the patch is awesome, it fixes all my av sync issues. Great work!
For those having problems like Herve, Mayank Jain and Jkb, you need to add in the -lavcore library into the makefile, so it now reads like:
all:
gcc -Wall -g live_segmenter.c -o live_segmenter -lavformat -lavcodec -lavutil -lavcore -lbz2 -lm -lz -lfaac -lmp3lame -lx264 -lfaad -lpthread
clean:
rm -f live_segmenter
@Rick It seems possible. I'll have to look at it more when I get time to work on the project again.
Hi Carson,
First of all thanks for all the work you've done here. Unfortunately I am having some issues getting it to work myself, I think most of the problems are due to new versions of VLC, Quicktime Broadcaster, and FFMpeg that I am using. I pulled the latest version of FFMpeg from git, and the configure line you have gives me :
When I remove the –enable-libfaad flag it gives me:
Upon taking a look in the ffmpeg directory, the latest version doesn't appear to have any faad / faac with it. I've been trying for weeks now to get something to work. Am I missing something? I don't understand why the latest version of ffmpeg would make the difference here.
I think I may have solved part of my problem.. looks like i just need to install faad, faac, and lame properly (i'm a linux noob so this has been hindering my progress since i have to search for everything) then re-configure ffmpeg. After I posted I went back a few comment pages and one of the responses does mention this. I'll keep working away. Thanks!
segmenter.c: In function `main':
segmenter.c:242: warning: `guess_format' is deprecated (declared at libavformat/ avformat.h:789)
/tmp/ccxSkN1c.o(.text+0xf): In function `add_output_stream':
/home/ankit/segmenter.c:28: undefined reference to `av_new_stream'
/tmp/ccxSkN1c.o(.text+0×563): In function `main':
/home/ankit/segmenter.c:177: undefined reference to `av_register_all'
/tmp/ccxSkN1c.o(.text+0×816):/home/ankit/segmenter.c:225: undefined reference to `av_find_input_format'
/tmp/ccxSkN1c.o(.text+0×858):/home/ankit/segmenter.c:231: undefined reference to `av_open_input_file'
/tmp/ccxSkN1c.o(.text+0×892):/home/ankit/segmenter.c:237: undefined reference to `av_find_stream_info'
/tmp/ccxSkN1c.o(.text+0x8ca):/home/ankit/segmenter.c:242: undefined reference to `guess_format'
/tmp/ccxSkN1c.o(.text+0x8fb):/home/ankit/segmenter.c:248: undefined reference to `avformat_alloc_context'
/tmp/ccxSkN1c.o(.text+0xa40):/home/ankit/segmenter.c:276: undefined reference to `av_set_parameters'
/tmp/ccxSkN1c.o(.text+0xa76):/home/ankit/segmenter.c:281: undefined reference to `dump_format'
/tmp/ccxSkN1c.o(.text+0xa8d):/home/ankit/segmenter.c:283: undefined reference to `avcodec_find_decoder'
/tmp/ccxSkN1c.o(.text+0xac0):/home/ankit/segmenter.c:288: undefined reference to `avcodec_open'
/tmp/ccxSkN1c.o(.text+0xb22):/home/ankit/segmenter.c:293: undefined reference to `url_fopen'
/tmp/ccxSkN1c.o(.text+0xb57):/home/ankit/segmenter.c:298: undefined reference to `av_write_header'
/tmp/ccxSkN1c.o(.text+0xbf5):/home/ankit/segmenter.c:309: undefined reference to `av_read_frame'
/tmp/ccxSkN1c.o(.text+0xc15):/home/ankit/segmenter.c:314: undefined reference to `av_dup_packet'
/tmp/ccxSkN1c.o(.text+0xc41):/home/ankit/segmenter.c:316: undefined reference to `av_free_packet'
/tmp/ccxSkN1c.o(.text+0xcdf):/home/ankit/segmenter.c:331: undefined reference to `put_flush_packet'
/tmp/ccxSkN1c.o(.text+0xcf0):/home/ankit/segmenter.c:332: undefined reference to `url_fclose'
/tmp/ccxSkN1c.o(.text+0xe1f):/home/ankit/segmenter.c:352: undefined reference to `url_fopen'
/tmp/ccxSkN1c.o(.text+0xe68):/home/ankit/segmenter.c:360: undefined reference to `av_interleaved_write_frame'
/tmp/ccxSkN1c.o(.text+0xeb7):/home/ankit/segmenter.c:366: undefined reference to `av_free_packet'
/tmp/ccxSkN1c.o(.text+0xecb):/home/ankit/segmenter.c:370: undefined reference to `av_free_packet'
/tmp/ccxSkN1c.o(.text+0xee3):/home/ankit/segmenter.c:373: undefined reference to `av_write_trailer'
/tmp/ccxSkN1c.o(.text+0xef4):/home/ankit/segmenter.c:375: undefined reference to `avcodec_close'
/tmp/ccxSkN1c.o(.text+0xf2a):/home/ankit/segmenter.c:378: undefined reference to `av_freep'
/tmp/ccxSkN1c.o(.text+0xf45):/home/ankit/segmenter.c:379: undefined reference to `av_freep'
/tmp/ccxSkN1c.o(.text+0xf60):/home/ankit/segmenter.c:382: undefined reference to `url_fclose'
/tmp/ccxSkN1c.o(.text+0xf6e):/home/ankit/segmenter.c:383: undefined reference to `av_free'
collect2: ld returned 1 exit status
make: *** [all] Error 1
I am getting the above error when I am trying to build the segmenter.
Any help will be appreciated
I'm live streaming from dreambox to ffmpeg. However the connection keeps resetting with "connection reset by peer". So I used wget in front of ffmpeg to use the capability of wget to retry connecting. It works ok but sometimes the pipe breaks and the segmenter does not function properly.
As a whole this system is unstable in terms of live streaming. Will check out vlc with httplive module.
PS. Unfortunately multiple streaming is not possible due to the fact that dreambox only accepts a single stream from its webserver.
Hey,
I'm trying to compile against the current ffmpeg HEAD & x264 HEAD, but I get this error:
http://ffmpeg.pastebin.com/wrs0pg1Y
Any clue how to proceed?
Hi all,
is a segmented stream seekable?
From what i've read here, the seeking doesn't work with HTTP Cupertino streaming, is it so?
With the various patches and tips here, everything is working great (thank you!), but I have noticed something odd that hopefully someone can shed some light on. When I try encoding at different bitrates (for the "adaptive" part of this) and segment the .ts files, the number of segments at each bitrate level don't match one-for-one. They should, right? I mean…they need to be the exactly the same, if I understand this correctly.
This becomes pretty obvious with longer clips. I might have 250 segments at bitrate and 270 segments at another.
Would the problem here be in the segmentation or the encoding?
If you get the message
then install yasm by
Also the formatting on this blog is messed up so view the source of this page to get the full command of configurations.
You can safely remove -lfaad from segmenter and ffmpeg building if you hit upon any issues with it.
Cheers
~gk
@Kevin I think the source needs an update for newer versions of ffmpeg. I have one in the works but not sure how long it will take to make sure it is working.
@Liano A live stream won't be seekable beyond the 3 preceding segments but a non-live stream can seek anywhere.
/usr/bin/ld: cannot find -llibavformat
collect2: ld returned 1 exit status
make: *** [all] Error 1
I have successfully compiled ffmpeg on Ubuntu. But when I am trying to compile the segmenter it is giving the above error. Googled on it but no specific solution. Can anyone please help on this.
Do I need to use any specific version of ffmpeg?
Does anyone have problem with the time offset ? what patch should I apply to avoid the time offset like the video end early but the audio still playing…etc?
I managed to get segmenter to compile on OSX, but all it did was segfault on my ts files (they are aac audio only, no video).
Then, I came across m3u8-segmenter. Hooray, works first time. Now if only I'd found it three hours ago…
Hi Carson ,
Thanks for such a nice tutorial it helped a lot.
But I am stuck, with playing .m3u8 file in Safari. I have taken the source code which you have given above.
But it is opening the index file in the browser as such instead of playing it. What could be wrong here . Any help will be appreciated.
This is a new one! I am also having trouble compiling the segmentor, but I have not seen the error I'm getting at the other people.
http://pastebin.com/d6hPVhhg
Help is much appreciated.
Thank you!
Getting the same error as Cristi with the git ffmpeg… Anyone know a version of the ffmpeg that works with this?
Here's my patch to get the original segmenter working against ffmpeg trunk. Note that m3u8-segmenter does not compile against ffmpeg trunk either.
===================================================================
— segmenter.c (revision 50)
+++ segmenter.c (working copy)
@@ -20,6 +20,20 @@
#include "libavformat/avformat.h"
+/* This code is here to allow old versions of ffmpeg to compile.
+** It should be removed in the near future after everyone updates
+** to the latest ffmpeg version.
+*/
+#define CodecType AVMediaType
+#define CODEC_TYPE_UNKNOWN AVMEDIA_TYPE_UNKNOWN
+#define CODEC_TYPE_VIDEO AVMEDIA_TYPE_VIDEO
+#define CODEC_TYPE_AUDIO AVMEDIA_TYPE_AUDIO
+#define CODEC_TYPE_DATA AVMEDIA_TYPE_DATA
+#define CODEC_TYPE_SUBTITLE AVMEDIA_TYPE_SUBTITLE
+#define CODEC_TYPE_ATTACHMENT AVMEDIA_TYPE_ATTACHMENT
+#define CODEC_TYPE_NB AVMEDIA_TYPE_NB
+#define PKT_FLAG_KEY AV_PKT_FLAG_KEY
+
static AVStream *add_output_stream(AVFormatContext *output_format_context, AVStream *input_stream) {
AVCodecContext *input_codec_context;
AVCodecContext *output_codec_context;
@@ -239,7 +253,7 @@
exit(1);
}
- ofmt = guess_format("mpegts", NULL, NULL);
+ ofmt = av_guess_format("mpegts", NULL, NULL);
if (!ofmt) {
fprintf(stderr, "Could not find MPEG-TS muxer\n");
exit(1);
ok just built the software and modified it to compile and link with the latest git of ffmpeg and x264 libraries
but i get a crash "***stack smashing …"
always when trying to start creating segment 40?
anybody got a version running with a new build of ffmpeg libs
also lots of memory lost
joolz
Dear Carson,
Is there a package with FFMpeg and stream segmenter which can be compiled, linked and built for an emmedded system (ARM, MIPS, x86 etc). Currently there seems to be a lot of dependency on a number of libraries. I got a link athttp://www.users.on.net/~belles/software/ffmpeg/
but this again does not contain the sources, but only libraries for a RISC architecture.
Regards
Sandeep Suresh.
Please note the svn mentioned above is quite old, and the latest version is herehttps://github.com/carsonmcdonald/HTTP-Live-Video-Stream-Segmenter-and-Distributor/tree/v2
I've just sent Carson pull request for minor changes fixing compilation against new versions of libav, not sure how active he is with it though.
Until applied, my fixed fork is here:
https://github.com/rdkls/HTTP-Live-Video-Stream-Segmenter-and-Distributor/tree/v2
Hi,
Are there any tools available to parse the master index file and the alternate index files for the URLs? I am working something that requires to parse the files and downloads the segments. I am hoping that there is a tool/API available to parse the index files.
Appreciate your help.
Thanks,
Padma.
i came across the same problem, a segmentation fault when hitting the 40th segment.
interestingly m3u8 segmenter has the same problem ( at segment 50 )
any ideas whats causing that?
Padma,
Such tools exist in ffmpeg trunk.
http://www.ffmpeg.org/doxygen/trunk/applehttpproto_8c.html
I hope they will include a muxer/segmenter in the future to complement the demuxer.
8 Trackbacks
[...] tutoriel de Ioncannon.net est un peu difficile à suivre. Pour simplifier la tâche des utilisateurs Mac OS 10.5, j'ai [...]
[...] Linux users as the new protocol is entirely based on open standards and can be implemented using open software. That will help erase the efforts to force video content into proprietary formats like WMV that [...]
[...] guides on the internet additionally force the number of reference frames down to 1 (ffmpeg's -refs [...]
[...] (seit FW 3.0) HTTPStreaming (Hier ein Link auf die Tech Specs). Auf ioncannon gibt es ein Tutorial wie man ein Video mit OpenSource Mitteln für gechunktes HTTP streaming [...]
[...] contar uma histórinha. Pesquisando sobre http streamming para iPhone eu cheguei ate esta solução open source que utiliza, entre outras coisas, o ffmpeg. Eis que chego até este [...]
[...] it turned out it is trickier than thought. Some useful command lines are shared here, here and here. One particular thing I ran into was the error "dts < pcr " which resulted in [...]
[...] blog and set of instructions for getting this to work is Carson McDonald's articles on iPhone HTTP Streaming with FFMpeg and an Open Source Segmenter and HTTP Live Video Stream Segmenter and [...]